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Section  1:  Overview 


I.A  Introduction 

Stottler  Henke  Associates,  Inc.  (SHAI)  was  awarded  the  Phase  I  SBIR,  N94-072,  Options 
to  Improve  Basic  Flight  Testing,  to  investigate  ways  that  Artificial  Intelligence  techniques  could 
be  used  to  enhance  flight  test  planning,  data  reduction,  and  reporting.  The  ultimate  goal  of  this 
project  was  to  reduce  the  time  required  to  develop  test  plans,  conduct  flight  testing,  analyze  flight 
test  data,  and  produce  reports. 

LB  Summary 

We  found  that  several  AI  techniques  would  be  applicable  to  the  development  of  a  system 
to  aid  flight  test  engineers  as  shown  in  the  figure  on  the  following  page.  We  investigated  several 
related  software  tools  and  found  that  many  would  not  be  useful.  FLIGHTLAB  was  certainly  an 
exception  in  this  regard.  Flight  test  engineers  reported  that  test  plan  and  test  report  creation  to  be 
their  most  time-consuming  activities  and  that  they  often  made  informal  use  of  similar  plans  and 
reports.  There  were  several  software  packages  of  which  they  made  extensive  use  which  would 
need  to  be  integrated  into  any  comprehensive  software  system.  Based  on  the  requirements  of  the 
flight  test  engineers,  we  developed  the  design  for  the  Automated  Flight  Test  Engineering  System 
(AFTES).  To  prove  its  feasibility  we  implemented  critical  portions  in  three  Phase  I  prototypes. 

I.C  Conclusions 

AFTES  is  feasible  and  will  benefit  the  Navy  substantially  when  it  is  implemented.  The 
three  prototypes  showed  that  the  most  critical  aspects  could  be  implemented.  Furthermore,  the 
very  positive  response  we  received  from  flight  test  engineers  over  both  the  AFTES  designs  and 
the  Phase  I  prototypes  indicates  that  AFTES  will  be  utilized  when  it  is  implemented.  Finally,  the 
fact  that  the  productivity  of  flight  test  engineers  will  markedly  improve  is  based  on  the  fact  that 
AFTES  was  designed  with  precisely  this  purpose,  along  with  improving  the  quality  of  the  plans 
and  reports  produced.  For  example,  the  test  plan  produced  in  a  few  minutes  by  the  prototype  can 
take  some  less-experienced  flight  test  engineers  several  days  to  produce. 
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-  Object-Oriented  Design 

-  Object-Oriented  Programming 


Section  II:  Detailed  Description  of  Results 

Section  II  provides  a  complete  and  detailed  description  of  the  analytic  results  which  led  to 
the  conclusions  stated  in  Section  I.  Subsection  A  gives  an  overview  of  the  Phase  I  effort. 
Subsection  B  describes  the  importance  of  automating  flight  test  engineers.  Subsection  C 
describes  the  methodologies  and  AI  techniques  used  in  Phase  I.  Subsection  D  gives  the  design  of 
AFTES.  Subsection  E  describes  the  Phase  I  prototypes. 

II.A  Phase  I  Overview 

The  Phase  I  effort  has  been  extremely  successful,  both  in  terms  of  proving  feasibility  of 
our  ideas,  and  user  acceptance  of  them.  After  the  Phase  I  demonstration,  we  received  several 
compliments  from  flight  test  engineers  that  our  solution,  the  Automated  Flight  Test  Engineering 
System  (AFTES),  would  be  usable  and  useful,  that  we  had  proven  our  ability  to  develop  AFTES, 
and  that  the  AFTES  capabilities  were  much  more  on  target  than  systems  proposed  by  previous 
researchers.  In  fact,  one  group  within  NAWCAD  PAX  RIVER  RW  will  operationally  use  one  of 
the  Phase  I  prototypes  to  retrieve  similar  test  points  as  part  of  the  Phase  I  effort. 

This  successful  Phase  I  conclusion  was  based  on  a  solid  foundation.  We  performed 
knowledge  engineering  by  examining  dozens  of  documents  and  had  several  in  depth  discussions 
with  flight  test  engineers  performing  a  variety  of  types  of  tests  throughout  Rotary  Wing.  Through 
intense  conversations  with  nine  different  flight  engineers  covering  the  areas  of  Attack  Assault,  Sea 
Control,  Dynamic  Interface,  R&M,  Flying  Qualities  and  Performance,  engine  testing,  simulator 
data  acquisition.  Structural  testing,  and  Avionics,  we  were  able  to  develop  user  requirements. 
From  the  user  requirements  we  designed  AFTES. 

We  surveyed  existing  related  software  tools  to  determine  which  might  be  useful.  We 
found  that  no  comprehensive  tools  existed  to  enhance  NAWCAD  PAX  RIVER  RW  flight  testing, 
but  did  identify  a  number  which  AFTES  should  interface  to.  We  then  implemented  the  most 
critical  aspects  of  AFTES  in  three  prototypes  to  absolutely  prove  feasibility  through  examples. 
These  three  prototypes  were  demonstrated  at  Pax  River  and  were  very  well-received.  The  first 
prototype  automatically  generated  DI  flight  test  plans  from  information  input  by  the  user  in  an 
intelligent  interface.  The  interface  had  fields  which  appeared  or  disappeared  based  on  previous 
responses.  The  user  could  input  which  ship  and  helicopter  would  be  used  as  well  as  which  types 
of  tests  should  be  performed,  definitions  of  the  appendices,  and  other  information  needed  for  the 
test  plan.  The  prototype  then  generated  a  complete  test  plan,  including  correct  paragraph,  figure, 
table,  and  appendix  references  (but  not  including  the  appendices  themselves).  The  plan  could  then 
be  input  and  edited  in  Microsoft  Word. 

The  second  prototype  retrieved  previous  DI  test  points  similar  to  a  target  test  point 
entered  by  the  user.  The  test  points  had  several  features  which  were  all  used  simultaneously. 
Conceptually,  all  fifteen  to  twenty  thousand  DI  test  points  were  simultaneously  examined  against 
all  of  the  features  of  the  target  case  and  the  most  similar  ones  retrieved  in  a  few  seconds.  The 
similarity  search  included  concepts  such  as  helicopters  (or  ships,  wind  speed,  etc.)  being  different 
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but  still  somewhat  similar.  This  similarity  search  prototype  was  so  well  received  that  we  agreed 
to  spend  another  couple  of  days  enhancing  it,  so  they  could  use  it  operationally. 

The  third  prototype  showed  an  example  of  using  the  SHAI  developed,  knowledge-based, 
planning  techniques  for  project  and  resource  scheduling.  About  twenty  flight  test  projects  were 
entered  and  scheduled  against  their  required  resources.  We  showed  the  effect  of  introducing  a 
new,  high  priority  test  project.  The  delays  caused  in  other  projects  could  be  anticipated  and 
solutions  developed  and  tried. 

II.B  Importance  of  Automating  Flight  Test  Engineering 

The  Navy  is  faced  with  the  problems  of  aging  fleet  helicopters.  In  the  current  budget 
environment,  dealing  with  fatigue  problems  and  upgrading  current  airframes  is  preferable  over 
new  airframe  acquisition.  For  NAWCAD  Pax  River  RW,  this  translates  into  lots  of  flight  testing 
for  new  equipment  and  dealing  with  the  problems  of  older  helicopters.  New  equipment  may 
consist  of  new  avionics,  upgraded  engines,  or  even  new  rotor  blades.  In  addition,  there  are 
several  different  types  of  testing  done  at  Pax  River,  including  Dynamic  Interface,  Avionics, 
Reliability  &  Maintainability  (R&M),  Flying  Qualities  &  Performance  (FQ&P),  Flight  Control 
Systems,  Engine,  Simulator  Data  Acquisition,  and  Structural  testing.  As  a  result  of  both  the 
diversity  of  the  testing  and  the  fact  that  it  is  primarily  performed  on  older  airframes,  NAWCAD 
Pax  River  has  some  unique  rotary  wing  testing  requirements. 

Meanwhile,  the  Navy  is  in  the  process  of  down-sizing  and  there  is,  in  essence,  a  hiring 
freeze.  But  the  testing  load  is  increasing  as  the  Navy  tries  to  keep  old  airframes 
operational  and  up  to  date.  There  are  hundreds  of  flight  test  engineers  both  within  Rotary  Wing 
and  the  more  inclusive  Naval  Air  Warfare  Center  Aircraft  Division  Pax  River.  The  largest 
percentage  of  their  time  is  spent  writing  test  plans  and  reports.  Even  a  modest  increase  in  their 
efficiency  will  save  the  Navy  enormous  sums  of  money.  Additionally,  there  is  a  need  to  preserve 
corporate  knowledge  before  the  engineers  who  possess  that  knowledge  leave  or  retire.  Once 
captured,  that  knowledge  can  be  shared  among  the  entire  group  of  engineers. 

The  need  to  preserve  corporate  knowledge  was  illustrated  by  an  actual  engineering 
requirement  during  Operation  Desert  Storm.  NAWCAD  PAX  RIVER  RW  was  tasked  with 
developing  and  testing  equipment  to  retrieve  airborne  Unmanned  Aerial  Vehicles  (UAVs)  using  a 
helicopter.  At  first  this  seemed  to  have  no  relation  to  any  previous  test.  But  one  of  the  older 
engineers  was  able  to  recall  that  during  the  1960's  helicopters  were  sometimes  used  to  retrieve 
film  canisters  re-entering  the  atmosphere  and  parachuting  from  spy  satellites.  The  engineering  and 
planning  reports  served  as  a  starting  point  for  the  current  study. 

A  system  to  automate  flight  testing  will  improve  test  plan  development  efficiency  and  test 
plan  quality  through  improved  consistency  within  and  between  plans,  improved  safety  and 
identification  of  hazards,  increased  use  of  lessons  learned  from  previous  tests,  and  by  allowing 
engineers  to  consider  problems  encountered  by  others.  Such  a  system  would  improve  the 
efficiency  of  the  approval  process  by  facilitating  the  use  of  already  approved  language  in  new 
plans  and  tracking  plan  changes  through  the  approval  and  collaboration  process.  The  system 
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would  improve  report  writing  efficiency,  report  writing  quality  and  consistency.  It  would  also  aid 
project  and  competency  scheduling. 

The  need  to  retrieve  similar  data  points  is  illustrated  by  a  more  recent  example.  In  order 
to  gather  data  for  a  training  simulator,  a  helicopter's  flight  control  system  was  configured 
unusually.  The  helicopter  then  performed  some  routine  control  response  tests.  Using  quarter 
inch  and  half  inch  longitudinal  stick  displacements  produced  no  unusual  results.  However  at  three 
quarters  of  an  inch,  the  helicopter  pitched  so  violently  forward  that  it  was  actually  slightly  upside 
down.  The  nose  angle  had  proceeded  from  0  to  more  than  ninety  degrees.  The  pilot  recovered 
and  was  able  to  make  a  safe  landing.  But  a  flight  test  engineer  planning  a  future  test  involving 
that  helicopter  with  similar  configuration  settings  should  retrieve  and  consider  that  extreme  test 
point  for  the  safety  of  all  involved. 

II.C  Phase  I  Methodologies 

While  we  approached  flight  test  engineering  with  a  number  of  innovative  ideas  and 
extensive  experience  in  the  field  of  AI  and  general  problem  solving,  we  did  not  wish  to 
inappropriately  impose  a  solution  or  methodology  on  the  problem.  Instead,  we  sought  to  first 
thoroughly  understand  the  complexities  of  flight  test  planning  and  reporting  and  the  needs  and 
requirements  of  the  flight  test  engineers.  Using  appropriate  AI  techniques,  we  then  tailored  a 
solution  to  the  problem.  The  AI  methodologies  involved  in  automating  flight  test  engineering 
include  Case  Based  Reasoning  (CBR),  knowledge  engineering  techniques,  knowledge 
representations.  Intelligent  Documents,  Intelligent  Interface,  AI  planning  methods  and  object 
oriented  programming.  Each  of  these  AI  techniques  is  described  in  the  following  subsections. 

II.C.  1  Case-Based  Reasoning 

SHAI  is  a  pioneer  in  the  development  and  application  of  Case-Based  Reasoning  (CBR). 
CBR  is  based  on  the  notion  that  people  often  solve  problems  by  remembering  the  solution  to  a 
similar  problem  and  adapting  that  solution  to  meet  the  current  circumstances.  CBR  mimics  this 
process.  CBR  is  relevant  to  several  capabilities  required  by  flight  test  engineers.  These  include 
plan  prototype  retrieval,  similar  plan  retrieval,  similar  test  point  retrieval,  hazards  and  lessons 
learned  for  similar  previous  test  projects,  and  similar  report  retrieval. 

Case-based  reasoning  is  a  knowledge  representation  and  control  methodology  based  upon 
previous  experiences  and  patterns  of  previous  experiences.  These  previous  experiences,  or 
"cases"  of  domain-specific  knowledge  and  action,  are  used  in  comparison  with  new  situations  or 
problems.  These  past  methods  of  solution  provide  expertise  for  use  in  new  situations  or 
problems. 

CBR  systems  offer  enormous  benefits  compared  to  standard  AI  approaches.  The 
knowledge  elicitation  bottleneck  is  largely  circumvented.  Cases  can  be  automatically  acquired 
directly  from  domain  experts.  Rules,  on  the  other  hand,  almost  always  require  the  intervention  of 
a  knowledge  engineer.  Instead  of  having  to  elicit  all  of  the  knowledge  required  to  derive  a 
solution  from  scratch,  only  the  knowledge  required  to  represent  a  solution  is  needed. 
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In  simple  applications,  a  case  might  be  represented  as  a  database  record  of  fields.  Only  the 
field  names  and  types  must  be  elicited.  The  data  can  be  entered  automatically.  So  knowledge 
elicitation  is  largely  avoided  with  CBR  and  may  be  COMPLETELY  automated  depending  on  the 
type  of  application  and  the  expert. 

Conventional  knowledge  base  technology  dictates  a  single,  fixed  problem  solving 
methodology.  With  CBR,  each  case,  in  the  extreme,  can  represent  a  different  methodology. 
Therefore,  many  problem  solving  methodologies  are  represented  and,  since  new  cases  are 
continually  added  automatically,  a  CBR  system's  problem  solving  methodologies  can  change  with 
time,  thus  improving  its  performance  and  staying  up  to  date  and  relevant  automatically. 

Much  of  the  research  in  Case-Based  Reasoning  is  directed  toward  retrieving  similar  cases 
and  determining  what  are  useful  definitions  of  similarity.  It  is  therefore  a  field  with  great  potential 
for  use  in  selecting  similar  test  plans,  test  points,  and  reports.  The  cases  are  simply  previously 
approved  plans  and  reports  and  previous  data  points,  from  which  inferences  and  comparisons  can 
be  made  using  CBR. 

The  interactions  between  the  different  factors  influencing  the  development  of  plans  and 
reports  for  a  given  flight  test  are  very  difficult  to  quantify  into  usable  rules  or  principles.  The  best 
way  to  make  these  evaluations  is  by  using  information  from  previous  cases  as  a  basis  for  the 
creation  of  new  plans  and  reports.  This  is  an  ideal  application  for  the  technique  of  Case-Based 
Reasoning 

The  use  of  CBR  solves  the  difficulty  described  above  with  appropriately  combining 
available  information  to  develop  a  plan  or  report.  The  relationships  between  combinations  of 
factors  will  be  represented  appropriately  in  the  newly  created  plans,  because  they  will  be  based  on 
similar  cases.  For  example,  if  the  type  of  helicopter  in  combination  with  the  type  of  testing 
strongly  affects  the  test  plan,  then  these  interactions  will  be  represented  in  the  test  plans  of  similar 
cases. 

II. C. 2  Speech  Recognition 

Speech  Recognition  systems  have  developed  to  the  point  that  they  are  leaving  the 
laboratory  and  being  fielded  in  applications.  Currently,  speech  to  text  dictation  systems  exist 
which  operate  at  about  60  words  per  minute.  That  is  faster  than  most  people  can  type  but  much 
slower  than  most  people  speak.  Most  of  these  systems  are  speaker  dependent  and  operate  on 
non-continuous  speech.  Speaker  dependent  means  that  the  system  must  be  trained  for  each 
individual  speaker  who  will  be  using  the  system,  which  takes  about  one  day.  Non-continuous 
means  that  the  speaker  must  pause  briefly  between  words,  which  can  be  unnatural  for  some. 

If  the  vocabulary  is  restricted  in  some  way,  then  speaker  independent,  continuous  speech 
systems  can  be  used.  This  may  occur  when  voice  is  used  as  an  alternative  for  menu  selection,  for 
example. 
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Recently,  researchers  have  reported  promising  results  with  speaker  independent, 
continuous  speech  dictation  systems.  The  research  is  currently  moving  very  fast,  driven  by 
expanding  use  in  the  commercial  marketplace.  It  is  likely  that  by  the  latter  half  of  this  Phase  II 
effort,  such  systems  will  have  been  validated  in  actual  applications  and  available  for  our  use. 

II.C.3  Knowledge  Engineering/Elicitation 

Knowledge  engineering  is  the  process  of  eliciting  and  organizing  information  from  experts 
in  a  particular  domain,  in  our  case,  flight  test  engineering,  and  is  the  necessary  precursor  to 
development  of  a  useful  software  prototype  or  tool.  Classic  AI  knowledge  engineering  was 
adapted  to  the  flight  test  engineering  domain  for  this  project.  We  have  performed  significant 
knowledge  engineering  in  Phase  I.  The  steps  in  knowledge  engineering  were: 

1 .  Met  with  several  different  engineers  to  discuss  the  details  of  planning  and  reporting.  The 
engineers  included  a  representative  sample  both  across  the  different  types  of  flight  tests  and 
different  levels  of  experience.  We  reviewed  previous  cases  of  planning  and  reporting  flight  tests, 
concentrating  on  those  projects  which  were  especially  difficult,  either  from  the  uniqueness  of  the 
test  or  from  the  engineer  not  having  enough  time.  This  process  was  made  easier  by  the  fact  that 
the  plans  and  reports  are  produced  in  relatively  standard  formats  but  also  more  difficult  by  the  fact 
that  the  process  of  developing  a  particular  plan  or  report  is  not  saved. 

2.  Observed  the  engineers  at  work.  We  asked  the  engineer  to  explain  why  he  included 
specific  tests  or  test  conditions  and  why  he  included  particular  analyses  in  the  report  of  test 
results.  We  asked  which  explanations  are  particular  to  the  current  project  and  which  are  more 
general. 

3.  Structured  the  test  planning  and  reporting  knowledge  obtained.  Discussed  the 
organization  of  the  knowledge  with  the  engineers. 

4.  Once  a  prototype  version  of  AFTES  had  been  developed,  presented  the  software  to  the 
engineers  and  to  get  their  feedback  on  the  correctness  and  optimality  of  the  produced  plans  and 
reports  and  solicit  suggestions  for  improvements. 

II .0.4  Knowledge  Representation 

In  order  to  automate  the  flight  test  planning  and  reporting  process  and  allow 
comprehensive  similar  test  project  retrieval,  one  must  first  establish  a  representation  in  the 
computer  of  the  test  project  and  its  components.  These  components  include  the  elements  of  the 
test  plan  such  as  the  tests,  test  methods,  test  conditions,  aircraft  and  equipment  descriptions, 
safety  checklist,  etc.;  elements  of  the  test  report  such  as  many  of  the  same  elements  as  the  test 
plan,  results,  conclusions,  deficiencies,  recommendations,  etc.;  and  the  data  collected  including 
the  analyses  and  presentation  methods  used. 

These  diverse  test  project  components  can  be  captured  using  AI  knowledge  representation 
techniques.  An  appropriate  knowledge  representation  is  one  that  naturally  and  completely 
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captures  desired  knowledge  in  the  domain  and  that  can  be  successfully  and  easily  manipulated  to 
meet  the  needs  of  the  application. 

Test  plans  and  reports  are  represented  as  objects  with  certain  attributes  or  features,  which 
can  be  used  to  find  them  in  similarity  retrievals.  Each  plan  or  report  object  includes  a  set  of 
objects  representing  individual  sections,  appendices,  budget,  and  other  parts  of  the  test  plan. 
These  individual  section  objects  also  may  be  individually  retrieved  in  a  similarity  search,  because 
even  if  the  entire  plan  is  not  relevant  to  the  current  situation,  an  individual  section  might  be  (such 
as  tests  methods  relative  to  a  certain  piece  of  avionics  equipment).  Section  objects  may  include 
subsection  objects  or  simply  the  text  from  the  plan  or  report. 

II. C. 5  Knowledge  Based  Planning 

An  important  aspect  of  many  AI  development  efforts  is  the  capture  of  the  corporate 
knowledge  of  the  experts.  By  eliciting  and  storing  the  details  of  a  process,  novices  can  be 
productive  even  when  the  experts  are  unavailable. 

The  required  knowledge  for  flight  test  planning  can  be  captured  in  a  number  of  ways. 

First,  the  expert's  knowledge  of  previous  test  projects  is  captured  as  a  collection  of  cases. 

Second,  the  expert's  knowledge  of  a  very  similar  set  of  test  plans  and  when  test  conditions  should 
be  included,  can  be  captured  using  test  condition  prototypes  or  intelligent  documents.  Rules  can 
be  used  to  capture  the  inclusion  criteria. 

II. C. 6  Object  Oriented  Programming 

Object  Oriented  Programming  (OOP)  is  a  methodology  for  both  representation  and 
programming.  Using  OOP  techniques,  one  can  define  different  types  of  objects  and  specialized 
program  methods  that  manipulate  them.  OOP  facilitates  the  concept  of  intelligent  documents  in 
flight  test  planning  and  reporting,  where  the  document  includes  the  intelligence  to  ask  users 
questions  and  tailor  itself  to  their  needs.  Each  object  used  in  the  plan  or  report  process  has  an 
associated  generation  method  and,  in  effect,  generates  itself,  triggering  the  generation  of  its  sub 
components  or  related  references.  For  example,  when  we  wish  to  generate  an  entire  test  plan,  we 
would  tell  the  test  plan  intelligent  document  object  to  use  its  generation  method.  This  method 
instructs  each  of  the  test  plan's  constituent  sections  and  appendices  to  generate  themselves.  Each 
section  would  then  send  generate  messages  to  each  of  its  subsections.  Each  subsection  would 
send  generate  messages  to  each  subsection  until  eventually  sentences  and  phrases  were  receiving 
generate  messages.  At  the  lowest  level  a  method  is  invoked  which  generates  text  based  on  user 
supplied  answers  or  knowledge  stored  in  the  system,  such  as  the  maximum  sideways  airspeed  of 
the  test  helicopter  or  calculating  the  paragraph  number  for  a  paragraph  reference.  The  concept  of 
intelligent  entities  allows  complex  generation  algorithms  to  be  built  from  very  simple,  particular 
ones. 


Rules  can  be  used  more  effectively  throughout  the  generation  process  because  they  can  be 
tailored  to  each  object's  generation  method.  Instead  of  supporting  a  very  large  rule  base  of 
general  rules,  we  provide  a  large  number  of  very  small,  specialized  rule  bases  associated  with  each 
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given  object's  generation  methods.  There  is  almost  no  interaction  between  the  rule  bases,  because 
they  are  only  related  to  the  intelligent  entity  (such  as  a  sentence)  to  which  they  are  attached, 

II. C.  7  Intelligent  Interface 

An  important  technology  to  ensure  user  acceptance  is  the  notion  of  intelligent  interfaces. 
One  example  is  the  intelligent  form.  When  the  system  requires  information  from  the  user,  it  is 
presented  as  fields  in  a  form.  This  gives  the  user  flexibility  to  fill  in  information  in  the  order  he 
wants  and  allows  him  to  see  a  field's  requested  information  in  the  context  of  other  fields,  which 
contrasts  sharply  with  the  more  traditional  expert  system  interface,  which  is  a  series  of  questions. 
Which  fields  are  displayed  changes  dynamically,  based  on  the  answers  to  other  questions  and 
related  knowledge  the  system  possesses.  For  example,  in  the  Phase  I  prototype  interface,  when 
the  user  entered  the  name  of  a  ship,  the  system  determined  the  ship's  class  using  knowledge  of  the 
Navy's  class  naming  conventions.  Most  ship  classes  are  either  RAST-equipped  or  not.  But  some 
classes,  like  FFG-61  contain  both  HAST  and  non-RAST  ships.  Only  when  the  user  has  entered  a 
ship  in  this  latter  class  will  the  system  display  a  field  asking  whether  the  particular  ship  is  RAST- 
equipped. 
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11.D  AFTES  Description 


Based  on  the  results  of  our  knowledge  engineering  efforts,  we  developed  a  set  of 
requirements  for  a  system  to  enhance  flight  testing.  We  then  designed  the  Automated  Flight  Test 
Engineering  System  (AFTES)  which  addressed  those  requirements.  AFTES  is  described  in  this 
section. 


The  Flight  Test  Engineer  (FTE)  has  several  different  responsibilities.  SHAI  is  striving  to 
aid  him  across  all  of  these  responsibilities  with  one  integrated  system.  This  "one  point  of  contact" 
for  the  FTE  will  necessarily  therefore  interface  to  several  other  tools.  It  will  also  contain  diverse 
functionality  which  is  divided  up  into  six  categories.  They  are  Test  Plan  Development,  Test  Plan 
Execution,  Data  Gathering,  Data  Reduction/Plotting/Presentation,  Results  Reporting  and 
Project/Competency  Management.  As  shown  in  the  following  Figure  1,  these  six  blocks  make  use 
of  several  external  tools  in  addition  to  their  own  internally  implemented  capabilities. 
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Figure  1 

The  eventual  AFTES  will  support  the  entire  acquisition  cycle  as  shown  in  Figure  2.  Early 
in  acquisition,  TEMPs  are  developed  in  ATPS  and  passed  down  to  AFTES  for  resource 
scheduling  and  flight  test  planning  in  support  of  developmental  testing.  After  this  phase  of  testing 
is  complete,  AFTES  then  supports  the  operational  testing  in  two  different  ways.  First,  the  plans, 
reports,  and  data  relating  to  testing  the  aircraft  during  developmental  testing  are  available  to  the 
operational  testing  community.  These  can  be  electronically  transferred  to  an  Operational  Test 
Planning  and  Reporting  System,  as  shown  in  Figure  2.  Second,  all  of  the  capabilities  of  AFTES 
are  available  and  can  be  applied  to  enhance  operational  flight  testing.  In  other  words  the 
Operational  Test  Planning  and  Reporting  System  may  be  based  on  AFTES,  itself 
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From  our  Knowledge  Engineering  and  previous  experience,  we  have  decided  that  the 
technologies  of  Case-Based  Reasoning  (CBR),  Intelligent  Documents,  Intelligent  Interfaces, 
Expert  System,  Object  Oriented  Design  &  Programming,  Planning  &  Scheduling,  and  natural 
language  processing  appear  to  be  good  solutions  for  much  of  the  functionality.  Most  of  the 
engineers  make  use  of  similar  plans  and  expressed  a  strong  desire  for  similarity  retrieval  to  be 
much  easier  and  comprehensive  than  currently  available. 

CBR  is  the  field  of  AI  dealing  with  the  retrieval  and  use  of  past  similar  experiences  (or 
cases)  to  solve  current  problems.  Each  case  base  consists  of  a  number  of  cases,  possibly  case 
prototypes,  one  or  more  definitions  of  similarity,  an  index,  and  similarity  retrieval  methods  as 
shown  in  Figure  3.  The  user  can  have  easy  access  to  the  definitions  of  similarity  in  order  to  tailor 
the  retrieval  to  his  current  desires.  Similarity  retrieval  is  generally  robust  without  being 
overwhelming.  Often  the  default  similarity  definition  will  be  quite  acceptable.  The  CBR  and 
other  AI  methodologies  are  described  in  detail  in  Section  II.C. 
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AFTES  will  run  on  PC  compatible  hardware,  under  Windows  and  on  Apple  Macintoshes. 
The  user  interface  will  be  graphical,  taking  mouse,  keyboard,  and  voice  input.  Some  external 
tools  will  be  incorporated  into  the  user  interface,  especially  word  processing  and  data  reduction 
tools.  AFTES  will  interface  to  other  engineering  tools  such  as  FLIGHTLAB,  CIFER,  ATPS,  etc. 
Networking  will  be  supported. 

II.D.l  Test  Plan  Development 

Test  Plan  Development  includes  several  capabilities  as  shown  in  Figure  4.  Most  involve 
the  creation  of  the  test  plan.  But  another  capability  will  involve  the  ability  to  simulate  or  rehearse 
a  plan,  to  find  problems  and  opportunities.  This  will  be  accomplished  by  interfacing  to  ART's 
FLIGHTLAB.  By  simulating  a  model  of  the  aircraft  using  the  Tests/Test  Conditions  matrix, 
aircraft  limits  can  be  checked,  data  can  be  generated  for  real-time  comparison  to  actual  tests,  and 
sensor  placement  checked. 

The  Test  Plan  creation  process  may  begin  by  examining  the  current  TEMP,  input  from 
ATPS.  Or  it  may  involve  the  input  of  an  AIRTASK  Order.  Information  about  the  helicopter  and 
other  equipment  mentioned  in  the  task  order  may  be  retrieved  from  the  appropriate  case  bases. 
The  descriptions  of  the  aircraft  and  its  mission  and  of  the  other  equipment  may  be  used  in  the  test 
plan. 
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A  large  part  of  the  plan  may  be  able  to  be  automatically  generated  at  this  point  with  the 
following  procedure.  This  aircraft  and  other  equipment  information  may  be  used  to  retrieve 
prototypical  plans  which  will  be  adapted  from  the  information  used  to  retrieve  them  as  well  as 
other  information  elicited  from  the  engineer.  Similarity  of  plans  in  this  case  would  be  based  on 
the  type  of  testing,  the  type  of  aircraft  and  the  types  of  other  equipment,  such  as  a  ship  or 
avionics.  The  near-final  plan  could  be  output  into  Word  or  WordPerfect  for  final  editing.  This  is 
the  approach  used  in  the  successful  Phase  I  prototype. 

Alternatively,  for  plans  with  no  current  matching  prototype,  similar  test  plans  (or  portions 
thereof)  can  be  retrieved  and  used  by  the  test  engineer  in  a  number  of  ways.  Similarity  can  be 
based  on  a  number  of  factors  including  similar  aircraft,  equipment,  test  objectives,  test  methods, 
hazards,  etc.  As  little  or  as  much  information  currently  defined  for  the  current  test  plan  can  be 
used  as  a  target  for  a  similarity  search.  Similar  previous  test  plans  can  be  used  for  many  purposes. 
The  previous  plan's  test  methods  may  serve  as  a  reminder  or  an  inspiration,  especially  if  innovative 
methods  were  used.  The  text  describing  the  same  test  methods  as  those  being  used  in  the  current 
plan  can  be  copied  over.  The  previous  plan  can  show  which  test  conditions  are  the  most 
appropriate  and  the  engineer  can  copy  over  tables  and  text  describing  them.  Most  importantly 
from  a  safety  perspective,  the  hazards  identified  in  the  previous  plans  can  be  used  as  a  starting 
point  to  identify  hazards  for  the  current  plan.  The  previous  plans  may  point  out  known  problems 
with  that  type  of  aircraft  or  testing  or  identify  new  problems.  Known  areas  which  do  not  need  to 
be  tested  may  be  described  in  the  previous  plan.  The  same  rationale  may  apply  to  the  current  test. 
References  to  test  standards  and  relevant  specifications  may  be  copied  from  a  previous  plan.  Data 
extraction  methods  used  in  a  previous  plan  may  be  appropriate  for  the  current  one.  The  current 
plan  may  have  similar  support  requirements.  For  Example,  perhaps  the  Operational  Security 
section  can  be  copied  and  adapted. 

The  engineer  may  also  retrieve  similar  test  points  in  one  of  two  ways.  He  may  examine 
the  test  points  associated  with  the  similar  test  plans  already  retrieved  or  he  may  retrieve  similar 
test  points  from  the  test  points  case  base.  The  test  points  case  base  will  be  fed  from  the  DI  Data 
Base,  the  Automated  Project  Notebook  in  Attack/Assault,  CAFTA  (the  data  base  of  test  points 
and  aircraft  configurations  used  by  the  ITT  for  the  V-22),  and  other  local  databases.  The  test 
points  will  be  linked  to  their  associated  test  plans,  reports,  and  the  methods  executed  on  them  for 
reduction,  plotting,  analysis,  and  presentation.  Similar  test  points  may  be  used  to  proceed  from 
the  known  to  the  unknown.  A  prudent  course  for  the  current  test  may  be  to  start  with  test  points 
taken  in  previous  similar  tests.  A  set  of  similar  test  points  from  a  previous  test  project  may  be 
verified  with  spot  checking,  thus  providing  more  data  for  the  current  analysis.  By  finding  out 
about  and  verifying  this  set  of  similar  points,  the  flight  test  engineer  can  avoid  duplication  of 
effort.  As  indicated  by  the  upside-down  helicopter  example  given  in  Section  5.3,  similar  previous 
test  points  may  be  extremely  important  for  finding  potential  problems  or  trouble  spots. 

Pax  River  has  a  diverse  set  of  very  talented  groups  to  support  all  aspects  of  flight  testing. 

As  several  flight  test  engineers  pointed  out,  if  you  have  a  need  for  some  expertise,  it  probably 
exists  at  Pax  River.  However  that  large,  diverse  set  makes  finding  the  one  particular  group 
needed  difficult.  A  case  base  of  engineering  capability  can  be  consulted  to  find  the  best  person  or 
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group  to  aid  the  engineer  with  some  specialized  aspect  of  his  test.  This  case  base  will  be  created 
and  indexed  by  capabilities  automatically  from  the  authors  of  the  test  plans  and  reports. 

Ships  are  a  special  case  of  equipment  so  the  ship  case  base  will  be  described  explicitly. 
Ships  will  be  judged  similar  within  their  class  and  classes  will  be  judged  similar  in  special 
circumstances  such  as  in  known  air  wake  similarities.  The  ship  objects  will  also  include  the  ship's 
description,  relevant  equipment,  and  other  interesting  parameters. 

The  engineer  may  now  have  a  fairly  complete  plan.  Some  plan  formats  may  benefit  from 
linked  sections  which  are  edited  in  parallel  to  keep  them  consistent.  Additionally,  structures 
engineers  may  require  significant  up  front  analysis  before  test  planning  is  complete  and  AFTES 
would  aid  them. 

Planning  methods  may  be  invoked  to  optimize  the  use  of  flight  resources  while  obeying 
constraints  in  the  gathering  of  required  data  points.  Automatic  scheduling  could  be  done  to 
resolve  resource  conflicts  between  projects.  Optionally,  the  groupings  used  in  similar  plans  can  be 
used  as  a  basis  for  grouping  test  conditions  and  maneuvers  into  flights.  The  system  could  then 
automatically  generate  flight  cards  in  a  tailored  format. 

AFTES  will  include  knowledge  of  specifications,  especially  those  specifications  which  are 
becoming  more  relevant  and  with  which  the  engineers  are  currently  less  familiar.  Examples  are 
MIL  83300  and  ADS-33. 

AFTES  will  support  the  contributory  test  plan  development  process,  which  will  become 
even  more  problematic  as  historically  separate  organizations  develop  combined  test  plans  instead 
of  separate  ones.  Additionally,  AFTES  will  support  the  editing  and  approval  process  which 
occurs  as  test  plans  make  their  way  up  the  chain  of  command.  The  capabilities  include  network 
access,  saving  an  audit  trail  of  what  changes  were  made  and  who  made  them,  and  the  display  of 
the  test  plans  with  different  fonts  or  colors  to  indicate  different  sources  of  changes. 

1I.D.2  Test  Plan  Execution 

During  text  execution,  similar  Test  Points,  Plans,  or  Reports  could  be  retrieved  from 
possibly  remote  testing  sites  to  help  explain  unexpected  results.  For  tests  run  in  locations  with  no 
available  communication  links,  smaller  standalone  versions  of  these  case  bases  can  exist. 

II.D.3  Data  Reduction/Piotting/Presentation/Analysis 

AFTES  will  aid  the  engineer  in  reducing  and  analyzing  his  data  in  a  number  of  ways  as 
shown  in  Figure  5.  It  could  serve  as  a  user-friendly  front-end  and  advisor  to  advanced  systems 
like  CIFER.  Many  engineers  could  use  a  frequency  analysis  advisor  which  referenced  the  ADS- 
33  Army  Specification.  Another  possibility  is  to  provide  advice  on  validating  simulation  models, 
which  is  often  done  for  trainers.  FLIGHTLAB  may  be  useful  here  as  well.  Software  which 
should  be  interfaced  to  includes: 
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CIFER 

ASAP 

RIPS 

EXCEL 

MatLab 

MathCad 

Sigma  Plot 

Origin 

USNTPS  Computer  Data  Reduction  Routines  by  J.  J.  McCue 


FLIGHTLA& 

CIFER - 

SIMULink  - 
USNTPS  - 

ASAP - 


Flight  Test  Engineer 


User  Interface 


EXCEL 


Origin 


Tool 

Knowledge 


Data 

Processing 

Methods 


Data  Reduction 
Knowledge  &  Equations 


Analysis  Knowledge 


Presentation 

Advisor 


Data  Reduction/Processing/Analysis/Presentation 


Figure  5 

Data  gathered  is  linked  both  to  the  test  plan  and  to  methods  which  manipulate  and 
transform  it.  By  storing  these  methods  with  the  data,  other  engineers  can  see  what  data  analysis 
steps  were  performed.  The  method  information  includes  the  software  and  its  version,  the  input 
files,  the  output  files,  and  which  functions  were  executed. 

The  Analysis  knowledge  would  include  knowledge  of  Bode,  Nichols,  and  Nyquist  plots, 
for  example.  This  knowledge  could  be  used  both  as  a  basis  for  recommendation  and  tutorial. 

II.D.4  Results  Reporting 

Results  Reporting  includes  several  modules  as  shown  Figure  6.  Reporting  of  test  results 
must  be  allowed  in  several  formats  since  some  groups  will  use  the  standard  and  others  will  report 
their  results  through  other  means.  In  the  case  of  a  standard  format,  much  of  the  report  can  be 
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generated  automatically  and  the  engineer  guided  through  the  remainder  with  the  following 
capabilities: 


-  Automatically  generating  each  section 

-  Copying  over  text  from  the  test  plan  and  changing  tense 

-  In  Results  and  Evaluation  Section,  generate  the  first  two  or  three  sentences 

-  Prompt  for  remaining  sentences 

-  Step  user  through  paragraphs  based  on  earlier  sections  of  the  test  plan 

-  Keep  paragraph  number  references  consistent 

-  Keep  language  consistent 

-  Link  paragraphs  so  they  are  changed  in  parallel 

-  Interface  to  word  processing,  plotting,  and  drawing  packages 


Word 


External  Software  & 
Data  Bases 


Wpf 


Flight  Test  Engineer 


User  Interface 


OTIC  - 

TID - ) 

Local - ) 

TRENDS - 

Army/NASA/Contractors— 

NASA  Research  DB - 

AHS  DRS - 


Test  Reports 
Case  Base 


Report 

Adaptation/ 

Generation 


Reporting 

Advisor 


Reporting 

Formats 

Knowledge 


Test  Plan  Use 
Knowledge 


Results  Reporting 


Figure  6 


II.D.5  Project/Competency  Management 

Managing  the  projects  and  competency  groups  can  be  greatly  aided  by  the  planning  and 
scheduling  technology  developed  by  SHAI  with  more  than  one  million  dollars  worth  of  research 
previously  funded  or  under  contract.  The  NAWCAD  PAX  RIVER  will  receive  the  benefits  of 
this  technology  for  no  extra  cost.  Project  scheduling  will  include  the  scheduling  of  manpower  and 
equipment  resources,  thus  finding  bottlenecks.  By  using  information  from  the  TEMPs  and  other 
sources,  future  project  workload  can  be  estimated  and  scheduled.  In  this  way,  potential  problems 
can  be  anticipated  and  averted.  A  project  manager  may  see  that  in  the  coming  months  there  will 
be  a  conflict  between  his  and  a  higher  priority  program  over  the  instrumentation  teams.  This 
allows  him  to  try  to  avoid  the  problem  by  attempting  to  move  up  his  instrumentation  requirements 
sooner. 
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Managers  of  the  competency  groups  would  benefit  from  the  same  kind  of  future 
scheduling  capability.  This  would  warn  them  of  over  or  under  utilization  problems  coming  down 
the  road  and  to  take  appropriate  action  such  as  hiring  more  engineers  or  finding  other 
competencies  for  temporary  assignment  of  his  engineers.  The  Competencies  would  also  benefit 
from  a  tutorial  component.  Previous  test  plans  can  be  examined  and  performed  again  in 
simulation.  The  simulation  models  can  also  be  used  to  give  the  student  better  qualitative  and 
quantitative  feel  for  the  helicopter  performance. 

II.D.6  Case  Bases 

Several  Case  Bases  have  been  referred  to  in  the  above  sections.  They  are  briefly  described 

here. 


Projects  Case  Base 

-  Projects  include;  Test  Plans,  Data,  Data  Processing  Methods,  Reports 

-  Link:  Test  plan  -  Data  gathered  -  Data  Processed  -  Final  report 

-  Keep  track  of  how  data  was  processed  (i  .e.  excel  spreadsheet)  so  the  processing 
could  be  duplicated 

-  Saving  the  process 

-  Retrieved  test  plan  is  more  valuable  if  user  can  see  the  results 

-  Links  between  projects,  some  projects  are  precursors  to  others 


Test  Plan  Case  Base 

-  Cases  originate  from  within  sections 

-  Test  Plan  Similarity 

Equipment,  its  function/mission 
Aircraft,  its  mission 

different  similarity  definitions  for  different  areas 

-  Includes: 

Test/Test  Conditions 
Sections 
Hazards 
AIRTASK 

Specs  Referenced/Compared  to 

-  Test  Plans  retrieved  in  whole  or  part 
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Results  Reports  Case  Base 

-  Cases  originate  from  TID,  DTIC,  and  from  within  groups 

-  Similarity  Retrieval 


Aircraft  Case  Base 

Aircraft  Descriptions/Illustrations  for  inclusion  in  plans  and  reports 
Features 

Similarity  between  Aircraft  types  predefined. 

Mission  descriptions  for  each 

Suggested  representative  tasks  and  tests  which  are  used  to  evaluate  the  aircraft's 
ability  to  do  them 
Envelopes  with  rationale 


Lessons  Learned  Case  Base 

Cases  originate  from  CALL/NALL  and  within  each  group. 


Hazards  Case  Base 

-  Cases  originate  from  test  plans  and  DOD  problem  reports 

-  Automated  Case-Base  analysis  to  suggest  common  ones  for  different  types  of 
tests  or  aircraft 

-  Historic  problems  from  formal  R&M  database  and  Cherry  Point 
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II.E  Detailed  Phase  I  Prototype  Description 

The  Phase  I  effort  included  development  of  three  prototypes  -  Automatic  Test  Plan 
Generation,  Similar  Test  Point  Retrieval,  and  Test  Project  Scheduling. 

lI.E.l  Automatic  Test  Plan  Generation 


The  first  prototype  automatically  generated  D1  flight  test  plans  from  information  input  by 
the  user  in  an  intelligent  interface  Shown  in  Figures  7  through  9.  The  interface  had  fields  which 
appeared  or  disappeared  based  on  previous  responses.  For  example,  in  Figure  7,  RAST-Equipped 
only  appears  if  a  ship's  name  is  entered  which  corresponds  to  a  class  of  ships,  some  of  which  are 
RAST-equipped  and  some  of  which  are  not.  Likewise,  which  kind  of  lighting  evaluation  will  be 
done  is  only  requested  if  the  user  inputs  that  a  lighting  evaluation  will  be  done  at  all.  The  user 
could  input  which  ship  and  helicopter  would  be  used  as  well  as  which  types  of  tests  should  be 
performed,  definitions  of  the  appendices,  and  other  information  needed  for  the  test  plan.  The 
prototype  then  generated  a  complete  test  plan,  including  correct  paragraph,  figure,  table,  and 
appendix  references  (but  not  including  the  appendices  themselves).  The  plan  could  then  be  input 
and  edited  in  Microsoft  Word.  The  Test  plan  generated  from  the  input  from  the  following 
screens  is  given  in  the  Appendix. 


AUTOMATIC  FLIGHT  TEST  PUN  GENEFlAtOR 


TEsnr  i%AN  mpo 


1 
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Test  ISnpfamstttt 
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Orm  \ 


Oy#* 
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Figure  7 
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The  Automatic  Test  Plan  Generation  prototype  was  developed  by  scanning  a  test  plan  into 
electronic  form  and  parsing  it  into  sections,  subsections,  and  sentences,  based  partly  on  the  table 
of  contents.  The  sentences  were  then  parsed  to  identify  ones  change  for  different  plans,  such  as 
helicopter  names,  or  paragraph  or  figure  references.  These  were  then  exchanged  for  intelligent 
objects  which  could  generate  the  appropriate  text  based  on  user  responses  or  other  calculations. 

11.  E.  2  Similar  Test  Point  Retrieval 


The  second  prototype  retrieved  previous  D1  test  points  similar  to  a  target  test  point 
entered  by  the  user.  The  test  points  had  several  features  (as  shown  in  Table  1)  which  were  all 
used  simultaneously.  Conceptually,  all  fifteen  to  twenty  thousand  D1  test  points  were 
simultaneously  examined  against  all  of  the  features  of  the  target  case  and  the  most  similar  ones 
retrieved  in  a  few  seconds.  Table  2  shows  the  output  from  the  similarity  retrieval  which 
corresponds  to  the  input  of  Table  1.  The  most  similar  retrieved  data  point  is  shown  in  Table  3. 
The  similarity  search  included  concepts  such  as  helicopters  (or  ships,  wind  speed,  etc.)  being 
different  but  still  somewhat  similar.  This  similarity  search  prototype  was  so  well  received  that  we 
agreed  to  spend  another  couple  of  days  enhancing  it,  so  they  could  use  it  operationally. 

Ship:  AGF-5 
Landing  Spot:  1 
Helicopter:  AH-IW 
Wind  Speed:  30 
Wind  Direction:  90 
Approach:  P 
Pitch:  4 
Roll:  8 

WMO  Sea  State:  4 
Visibility:  4 

Target  Test  Point  Input 
Table  1 


Score 

Helicopter 

Ship-Hull-Spot  W.Dir 

W.  Speed 

Target 

AH-IW 

AGF-3-1 

90 

30 

56% 

AH-IT 

LPD-4-1 

85 

30 

56% 

AH-IW 

LHD-1-8 

95 

35 

55% 

AH-IW 

LHD-1-1 

80 

30 

54% 

AH-IW 

LHD-1-8 

95 

30 

54% 

AH-IW 

LHD-1-1 

100 

30 
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Similarity  Retrieval  Output 
Table  2 

Landing:  1  =  56% 

Helicopter  =  AH- 1 T 
Ship  =  LPD-4 
Landing  Spot  =  1 
Hull  Number  =  4 
Wind  Speed  =  30 
Wind  Direction  =  85 
Approach  =  P 
Pitch  =  5 
Roll  =  9 

WMO  Sea  State  =  2 
Visibility  =  1 
Time  =  3:0 
Longitude  =  159 
Latitude  =  67 
ACCG=  145 
ACGW=  10747 
Take  OffPRS  =  3 
Recovery  PRS  =  4 
Desired  LU  =  0 
Actual  LU  =  8 

Most  Similar  Retrieved  Point 
Table  3 

1I.E.3  Test  Project  Scheduling 

The  third  prototype  showed  an  example  of  using  the  SHAI  developed,  knowledge-based, 
planning  techniques  for  project  and  resource  scheduling.  About  twenty  flight  test  projects  were 
entered  and  automatically  scheduled  against  their  required  resources.  The  resulting  schedule  is 
shown  in  Figures  10  and  11.  We  then  introduced  a  new,  high  priority  test  project  and 
automatically  rescheduled.  The  resulting  schedule  is  shown  in  Figures  12  and  13,  and  reflects  the 
effects  of  the  high  priority  project  grabbing  resources  from  the  other  projects.  The  resulting 
delays  caused  in  these  other  projects  could  be  anticipated  and  solutions  developed  and  tried. 
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II.F  Innovations  of  the  AFTES  Design 

There  are  several  aspects  of  AFTES  development  which  are  especially  innovative.  They 
are  described  below. 

1.  Comprehensive,  Integrated  Treatment  of  Flight  Test  Engineering 

In  developing  the  AFTES  design,  SHAI  investigated  all  aspects  of  flight  test  engineering, 
not  just  one  narrow  focus.  We  investigated  several  flight  test  engineering  tasks  and  several  types 
of  flight  testing,  to  design  a  comprehensive,  integrated  system  which  would  aid  the  engineer 
across  all  of  his  tasks  and  across  several  testing  domains. 

2.  Use  of  Several  AI  Techniques  in  Combination 

SHAI  prides  itself  in  knowledge  of  a  diverse  set  of  Artificial  Intelligence  solution 
techniques.  With  this  knowledge,  we  examined  the  various  aspects  of  the  flight  testing  process 
and  developed  a  solution  which  addresses  each  task  with  the  technology  most  suited.  The 
AFTES  solution,  therefore,  includes  several  different  techniques.  These  include  Case  Based 
Reasoning  (CBR)  ,  natural  language  processing,  Object-Oriented  Design  and  Programming, 
Intelligent  Documents,  Intelligent  Interfaces,  Knowledge  Representation,  conventional  expert 
system  technology,  and  speech  recognition. 

3.  Case  Based  Reasoning 

Case  Based  Reasoning  (CBR)  has  not  been  applied  to  flight  testing  before.  CBR  systems 
can  adapt  by  learning  new  or  innovative  methods,  simply  through  the  addition  of  more  cases  (test 
plans  and  reports).  These  cases  will  include  sub  cases  which  can  be  retrieved  corresponding  to 
sections  of  the  plans  and  reports.  CBR  will  formalize  the  current  informal  practice  of  using 
similar  plans  and  reports.  By  capturing  test  projects,  these  projects  are  preserved  and  can  be 
shared  among  all  users,  when  appropriate.  Similarity  retrieval  will  be  supported  for  similar  test 
plans,  test  reports,  and  test  points.  This  capability  replaces  the  more  unwieldy  use  of  Boolean  or 
relational  data  base  queries.  Instead  of  specifying  the  desired  data  with  logical  clauses  connected 
with  Ands  and  Ors,  the  user  specifies  his  current  situation,  and  similar  items  are  retrieved.  He  can 
easily  change  the  relative  importance  of  various  features.  This  type  of  retrieval  is  much  more 
natural  since  it  more  closely  approximates  the  functioning  of  human  memory.  A  final  benefit  of 
CBR  technology  is  that  the  captured  cases  can  be  used  as  a  basis  for  tutoring. 

4.  Automatic  Test  Plan  and  Test  Report  Generation 

Often,  most  of  a  test  plan  or  test  report  can  be  automatically  generated  from  a  prototype 
plan  or  report.  A  single  prototype  represents  a  set  of  plans  and  reports  by  listing  which  parts  they 
have  in  common  and  describing  the  differences  along  with  conditions  on  when  those  different 
parts  are  applicable.  Using  the  conditions  and  user  input,  the  plan  or  report  is  generated  and 
output  in  word  processing  form  for  final  editing  by  the  user.  We  successfully  demonstrated  this 
capability  for  Dynamic  Interface  test  plans  in  Phase  I. 

5.  Intelligent  Documents 

An  object-oriented  design  technique  is  to  group  procedural  code  along  with  the  data  on 
which  the  code  operates  together  into  objects.  This  can  be  taken  one  step  further  by  representing 
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with  the  object,  all  of  the  knowledge  required  to  manipulate  the  object,  leading  to  the  concept  of 
intelligent  entities.  One  specialization  of  the  intelligent  entity  concept  is  intelligent  documents, 
where  a  document  includes  all  of  the  knowledge  required  to  ask  the  user  for  input  and  tailor  itself 
to  the  user's  current  requirements. 

6.  Intelligent  Interfaces 

An  intelligent  interface,  alters  the  set  of  required  input  from  the  user  based  on  previous 
user  inputs.  In  expert  systems,  this  was  traditionally  done  in  a  question  and  answer  (consultation) 
format,  where  a  user  answered  questions  which  were  presented  to  him  one  at  a  time.  A  more 
user-oriented  approach  is.  to  supply  a  dynamic  form,  whose  fields  change  as  the  user  provides 
input  to  other  ones.  In  this  way,  the  user  can  select  the  order  of  input  and  sees  one  field  in  the 
context  of  the  others. 

7.  Semi-Automated  Document  Prototype  Creation 

In  Phase  I,  SHAI  demonstrated  the  feasibility  of  generating  a  prototype  test  plan  from  an 
actual  one.  The  test  plan  was  scanned  into  electronic  form,  parsed  into  sections,  subsections,  and 
sentences,  based  partly  on  the  table  of  contents.  The  sentences  were  then  parsed  to  identify  ones 
which  change  for  different  plans,  such  as  helicopter  names,  or  paragraph  or  figure  references. 
These  were  then  exchanged  for  intelligent  objects  which  could  generate  the  appropriate  text  based 
on  user  responses  or  other  calculations.  This  process  will  also  be  applied  for  the  creation  of  test 
report  prototypes. 

8.  Test  Plan  Rehearsal 

By  driving  the  FLIGHTLAB  helicopter  simulation  with  the  test  condition  matrix,  a  test 
plan  can  be  rehearsed  and  possible  problems  found,  such  as  dangerous  test  conditions  or 
suboptimal  sensor  placement.  Additionally,  data  can  be  generated  for  real-time  comparison, 
leading  to  increased  safety. 

9.  Intelligent  Entity  Planning 

The  automated  techniques  SHAI  has  and  is  developing  for  knowledge-based  planning, 
funded  for  over  one  million  dollars  by  other  clients,  will  be  applied  to  project  and  resource 
scheduling  in  this  project  at  no  extra  cost  to  the  Navy. 
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II.G  Related  Systems 

Related  systems  have  been  developed  by  others.  FLIGHTLAB  is  currently  under 
development  by  Advanced  Rotorcraft  Technology  (ART)  located  near  SHAI.  FLIGHTLAB  is  a 
comprehensive,  flexible,  high-fidelity,  advanced  aerodynamic  simulation  capability  which  supports 
the  full  life-cycle  of  weapon  system  acquisition  from  design  thorough  testing  and  equipment 
updates.  SHAI  will  work  closely  with  ART  to  supply  AFTES  users  with  an  interface  to 
FLIGHTLAB.  SAIC  (Science  Applications  International  Corporation)  has  developed  the 
Automated  Test  Planning  System  (ATPS)  for  the  development  of  TEMPs  (Test  &  Evaluation 
Master  Plans).  AFTES  will  be  able  to  read  these  TEMPs  electronically. 

Test  PAES  (Test  Planning  Automation  and  Evaluation  System)  was  originally  developed 
for  human  factors  testing  at  Wright  Patterson  AFB,  but  has  been  expanded  to  include  others. 
TEST  PLAN  was  developed  by  G&C  Software  for  fighter  planes.  It  has  recently  been  applied  to 
new  fixed  wing  passenger  and  transport  aircraft.  These  systems  do  not  really  address  any  issues 
faced  by  Pax  River  flight  test  engineers. 

TP  AS  (Test  Planning  Automation  System)  was  a  prototype  developed  at  Pax  River  by 
John  McMaster.  It  much  more  closely  addresses  Pax  River's  needs  than  the  other  systems, 
especially  in  that  it  generated  the  text  of  a  test  plan.  Unfortunately,  funding  was  never  available  to 
move  it  beyond  the  prototype  stage. 
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Appendix:  Output  of  Automatic  Test  Plan  Generation  Prototype 


TABLE  OF  CONTENTS 

BACKGROUND 

PURPOSE 

DESCRIPTION  OF  TEST  EQUIPMENT 
UH-INHUEY 
CH-46E  SEA  KNIGHT 
USS  INGRAHAM  (FFG  61) 

ELEVATED  FIXED  PLATFORM  (EFP) 

NVD  COMPATIBLE  SHIPBOARD  LIGHTING 
SCOPE  OF  TESTS 

TESTS  AND  TEST  CONDITIONS 
TEST  LOADINGS 
TEST  ENVELOPES 
TEST  CONFIGURATIONS 
DATA  COLLECTION 
METHOD  OF  TESTS 

LANDBASED  FLIGHTS 
SHIPBOARD  FLIGHT  TESTS 
SHIPBOARD  NON-FLIGHT  TESTS 
INSTRUMENTATION 
SUPPORT  REQUIREMENTS 
SPECIAL  PRECAUTIONS 
MANAGEMENT 

FUNDING  REQUIREMENTS 
SCHEDULE/MILESTONES 
PERSONNEL  ASSIGNMENT 
PROJECT  SECURITY 
REPORTS 

PROJECT  DOCUMENTATION 
ENVIRONMENTAL  IMPACT 
LIST  OF  APPENDICES 

BACKGROUND 

1  Reference  (a)  tasked  NAVAIRWARCENACDIV  Patuxent  River  MD  (NAWCAD  Pax 
River)  to  conduct  dynamic  interface  (DI)  testing  of  various  helicopter/ship  combinations,  in  order 
to  improve  all  aspects  of  shipboard  helicopter  compatibility,  and  to  develop  shipboard  operational 
launch/recovery  envelopes.  References  (b)  and  (c)  reported  the  fleet  need  for  development  of  UH- 
IN  and  CH-46E  launch/recovery  operational  envelopes  aboard  Recovery  Assist,  Secure  Traverse 
(RAST)-equipped  FFG  7  class  ships.  References  (d)  through  (m)  discussed  the  availability  of  ship, 
helicopter,  and  maintenance  assets  in  support  of  UH-IN  and  CH-46E  shipboard  high  sea  state  DI 
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launch/recovery  envelope  development  tests  conducted  aboard  USS  INGRAHAM  (FFG  61)  in 
March  1  to  March  15,  1995. 

2.  Reference  (n)  tasked  NAWCAD  Pax  River  to  conduct  flight  test  and  evaluation  efforts  in 
order  to  develop  shipboard  lighting  and  marking  configurations  that  are  compatible  with 
rotorcraft  Night  Vision  Device  (NVD)  operations  aboard  various  air-capable  surface  combatant 
ships.  References  (o)  and  (p)  reported  results  of  shipboard  H-60/flight  deck  lighting  NVD 
compatibility  evaluations  conducted  under  this  tasking,  and  recommended  the  conduct  of 
additional  shipboard  flight  test  operations  to  further  validate  the  concept.  Reference  (q)  confirmed 
the  desirability  of  conducting  a  night  NVD  and  non-NVD  shipboard  lighting  compatibility 
evaluation  concurrently  with  previously  scheduled  UH-IN  and  CH-46E  test  operations  in  early 
1994. 

3.  As  discussed  in  references  (h)  through  (r),  in  support  of  the  test  efforts  described  in 
paragraphs  1  through  3,  at-sea  UH-IN  and  CH-46E  DI  tests  will  be  conducted  aboard  USS 
INGRAHAM  (FFG  61)  between  March  1  to  March  15,  1995,  during  transit  from  Norfolk,  VA  to 
Jacksonville,  FL. 

PURPOSE 

4.  The  primary  purpose  of  this  evaluation  to  develop  UH-IN  and  CH-46E  day/night  (NVD 
and  non-NVD)  nondegraded  Automatic  Flight  Controls  System  (AFCS)  shipboard 
launch/recovery  envelopes  aboard  RAST-equipped  FFG  7  class  ships.  Secondary  purposes 
include; 

a.  NVD  and  non-NVD  evaluation  of  the  compatibility  characteristics  of  a  prototype  NVD- 
compatible  flight  deck.  Helicopter  Control  Station  (HCS),  and  LSO  (Landing  Safety  Officer) 
station  lighting  configuration  for  use  by  various  aircraft  aboard  FFG  7  class  ships. 

b.  Documentation  of  RAST-equipped  FFG  7  class  ship  motion  characteristics. 
DESCRIPTION  OF  TEST  EQUIPMENT 

UH-IN  HUEY 

5.  The  UH-IN  is  a  utility  transport  helicopter  manufactured  by  Bell  Helicopter  Textron  for 
the  United  States  Marine  Corps,  The  aircraft's  maximum  gross  weight  is  10,500  lb.  UH-IN 
airspeed  limits  are  80  KIAS  for  forward  flight,  30  KTAS  for  sideward  flight,  and  10  KTAS  for 
rearward  flight.  A  detailed  description  of  the  UH-IN  is  found  in  reference  (s);  a  drawing  of  the 
aircraft  is  presented  in  appendix  B,  figure  1.  An  RWATD  UH-IN  will  be  used  for  landbased 
buildup  evolutions,  and  an  embarked  fleet  UH-IN  will  be  used  for  all  shipboard  evolutions.  The 
shipboard  test  aircraft  and  maintenance  support  will  be  provided  by  an  embarked  UH-IN 
detachment  from  Marine  Light  Attack  Helicopter  Squadron  (HMLA),  from  MCAS  Camp 
Pendleton,  CA,  The  UH-IN  test  aircraft  will  be  capable  of  shipboard  NVD  operations,  and  if 
possible,  will  carry  internal  auxiliary  fuel  tanks.  If  possible,  the  UH-IN  will  be  equipped  with  a 
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stab  bar  (vice  a  SCAS-equipped  aircraft)  to  test  worst  case  handling  qualities.  Both  UH-lNs  will 
be  representative  of  production  UH-IN  aircraft  for  test  purposes. 

CH-46E  SEA  KNIGHT 

6.  The  CH-46E  is  a  tandem  rotor  assault  transport  helicopter  manufactured  by  the  Boeing 
Vertol  (now  Boeing  Helicopter)  Company  for  the  United  Stated  Marine  Corps.  The  aircraft's 
maximum  gross  weight  (internal  and/or  external  cargo)  is  24,300  lb.  CH-46E  airspeed  limits  are 
145  KIAS  for  forward  flight,  35  KTAS  for  sideward  flight,  and  30  KTAS  for  rearward  flight.  A 
detailed  description  of  the  CH-46E  is  found  in  reference  (s);  a  drawing  of  the  aircraft  is  presented 
in  appendix  B,  figure  1 .  An  RWATD  CH-46E  will  be  used  for  landbased  buildup  evolutions,  and 
an  embarked  fleet  CH-46E  will  be  used  for  all  shipboard  evolutions.  The  shipboard  test  aircraft 
and  maintenance  support  will  be  provided  by  an  embarked  CH-46E  detachment  from  Marine 
Medium  Helicopter  Squadron  (HMM),  from  MCAS  Tustin,  CA.  The  UH-IN  test  aircraft  will  be 
capable  of  shipboard  NVD  operations,  and  if  possible,  will  carry  internal  auxiliary  fuel  tanks.  Both 
CH-46ES  will  be  representative  of  production  CH-46E  aircraft  for  test  purposes. 

USS  INGRAHAM  (FFG  61) 

7.  USS  INGRAHAM  (FFG  61)  is  the  fifty-first  and  last  (commissioned  5  August  1989)  ship 
of  the  Oliver  Hazard  Perry  (FFG  7)  class  of  guided  missile  fast  frigates.  Built  by  Todd  Shipyards, 
San  Pedro,  CA,  USS  INGRAHAM,  like  each  ship  in  the  FFG  7  class,  is  453  ft  long,  45  ft  wide, 
has  a  25  ft  draft,  and  displaces  approximately  4100  tons  fully  loaded.  Two  General  Electric  LM- 
2500  marine  gas  turbine  engines,  producing  a  total  of  approximately  41,000  shp,  propel  each  FFG 
7  class  ship  via  a  single,  constant  pitch  screw  to  speeds  in  excess  of  30  kt.  Each  FFG  7  class  ship 
has  a  single  rudder  and  fin  stabilizers  to  improve  roll  stability  (usually  at  the  expense  of  an 
increased  vertical  or  heave,  motion).  Each  ship  in  the  FFG  7  class  has  a  single  helicopter 
operations  platform,  which  is  located  immediately  aft  of  twin  hangars  (each  capable  of  holding  a 
folded  H-60  or  equivalent  sized  aircraft)  and  approximately  1 6  ft  above  the  ship's  waterline.  On  all 
FFG  7  class  ships  completed  after  FFG  36  (including  USS  INGRAHAM),  and  also  as  a  retrofit  to 
several  earlier  ships,  the  aft  portion  of  the  flight  deck  has  been  lengthened  and  two  complete  (one 
to  port,  and  one  to  starboard)  RAST  systems  have  been  installed,  permitting  H-60  (or  suitably- 
equipped  derivative)  operations  in  adverse  ship  motion  conditions.  An  aviation  facilities  summary 
and  flight  deck  drawing  for  USS  INGRAHAM  is  presented  in  appendix  B,  figure  3.  Reference  (u) 
provides  the  most  recent  aviation  facilities  certification  for  USS  INGRAHAM.  A  description  of 
the  standard  RAST-equipped  FFG  7  class  Visual  Landing  Aids  (VLA)  lighting  package  (installed 
aboard  USS  INGRAHAM)  is  presented  in  appendix  I,  table  I. 

ELEVATED  FIXED  PLATFORM  (EFP) 

8.  The  Elevated  Fixed  Platform  (EFP)  Facility,  located  at  Naval  Air  Warfare  Center, 
Lakehurst,  NJ  (NAWCAD  Lakehurst),  is  a  reconfigurable  fixed  platform  that  currently  simulates 
an  FFG  7  class  ship  flight  deck  and  aft  hangar  face.  It  includes  operational  FFG  7  class  Visual 
Landing  Aids  (VLA),  including  deck  and  hangar  face  lighting.  The  EFP  maintains  an  operable 
LSO  Station  (moved  laterally  outboard  from  its  actual  shipboard  location  for  safety  reasons),  and 
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fully  functional  RAST  hauldown,  securing,  and  traversing  equipment,  thereby  producing  a 
realistic  land-based  RAST  system  virtually  identical  to  that  of  a  RAST-equipped  ship.  The  EFP 
does  not  move,  and  thus  cannot  duplicate  wind  or  ship  motion  conditions  typical  of  ships,  but 
otherwise,  exhibits  excellent  fidelity  for  use  in  pilot/LSO  training,  proficiency,  and/or  other  limited 
scope  evaluation.  NAWCAD  Pax  River  has  used  the  EFP  for  over  10  prior  H-60  D1  buildup 
evolutions  within  the  last  3  years.  During  pilot  buildup  evolutions  conducted  as  part  of  this  test, 
the  EFP  will  be  configured  with  the  same  NVD  compatible  lighting  configuration  that  will  be 
employed  during  this  at-sea  test.  Appendix  H  contains  drawings  and  photographs  of  the  EFP. 

NVD  COMPATIBLE  SHIPBOARD  LIGHTING 

9.  As  discussed  in  references  (o)  and  (p),  typical  RAST-equipped  DD  963,  CG  47,  or  FFG  7 
class  shipboard  flight  deck  VLA  lighting  configurations  are  not  compatible  with  NVD  helicopter 
operations.  However,  as  discussed  in  references  (o)  and  (p),  the  NVD  compatibility 
characteristics  of  a  modified  flight  deck  VLA  lighting  configuration  were  evaluated  aboard  USS 
RENTZ  (FFG  46)  and  aboard  USS  ARTHUR  W.  RADFORD  (DD  968)  in  1992  and  1993.  This 
VLA  lighting  configuration,  described  in  appendix  I,  table  II,  consists  of  blue  filters  installed  in  the 
overhead  floodlights  and  blue  filters  and  cool  beam  lamp  units  installed  in  the  hangar  face  and 
deck  surface  floodlight  (dustpan)  units.  Based  on  FFG  46  and  DD  968  DI  test  results  provided  in 
references  (o)  and  (p),  additional  ship's  interior  lighting  NVD  compatibility  is  provided  by  the 
addition  of  several  Electro-luminescent  (EL)  panels  mounted  in  both  HCS  and  LSO  stations.  This 
shipboard  lighting  configuration  (which  previously  has  demonstrated  the  capability  to  provide 
ample  flight  deck  and  aircraft  illumination,  suitable  for  use  by  both  non-NVD  equipped  shipboard 
personnel  and  by  NVD  or  unaided  aircrews  conducting  operations  to  such  decks)  will  be 
evaluated  throughout  all  night  tests  described  in  this  document. 

SCOPE  OF  TESTS 

TESTS  AND  TEST  CONDITIONS 

10.  A  summary  of  landbased  and  shipboard  test  priorities  is  presented  in  appendix  D,  table  I. 

To  successfully  complete  the  desired  test  objectives,  the  evaluations  described  in  appendix  D, 
table  II  will  be  conducted  aboard  USS  INGRAHAM  (FFG  61)  during  transit  from  Norfolk,  VA 
to  Jacksonville,  FL  on  March  1  to  March  15,  1995.  At-sea  UH-IN  and  CH-46E  testing  will 
include  both  day  and  night  test  periods:  approximately  six  day  and  six  night  shipboard  flight  test 
periods  (of  approximately  3-4  hours  duration  each,  producing  a  total  of  approximately  40 
shipboard  flight  test  hours)  are  scheduled.  Tentative  plans  include  the  use  of  embarked  fleet 
aircraft  and  maintenance  support  for  all  shipboard  flight  test  operations. 

11.  In  accordance  with  appendix  D,  table  I  through  III,  shipboard  operations  will  primarily 
consist  of  at-sea  day  and  night  (NVD  and  non-NVD)  UH-IN  and  CH-46E  launch/recovery 
evolutions.  The  tentative  plan  depicted  in  appendix  D,  tables  I  through  III  is  designed  to  facilitate 
the  concurrent  accomplishment  of  all  test  objectives  aboard  up  to  three  different  ships  during  up 
to  three  different  underway  periods.  Additional  or  alternative  test  periods  will  be  scheduled  as 
available;  the  number  and  duration  of  test  periods,  as  well  as  the  daily  flight  schedule,  will  be 
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modified  as  necessary  after  the  start  of  testing,  dependent  on  weather,  scheduling,  test 
completion,  crew  rest,  or  other  emergent  requirements. 

TEST  LOADINGS 

12.  All  test  flights  will  be  conducted  within  gross  weight  and  CG  limits  defined  in  the 
appropriate  NATOPS  and  NWP  42  (Shipboard  Helicopter  Operational  Procedures)  manuals 
(references  (s),  (t),  and  (x),  respectively).  During  shipboard  flight  operations,  the  aircraft  will  be 
refueled  aboard  ship  approximately  every  two  hours  as  necessary  to  achieve  the  desired  loadings 
specified  in  appendix  D,  table  II. 

TEST  ENVELOPES 

13.  Test  flights  will  be  conducted  within  limitations  described  in  the  appropriate  aircraft 
NATOPS  Flight  Manuals  (references  (s)  and  (t)).  Shipboard  Helicopter  Operational  Procedures 
Manual  (NWP  42  Rev  J.  reference  (x)),  CNSP  NVD  SOP  (reference  (x),  and  RWATD  SOP 
(reference  (y)),  except  as  authorized  by  the  DI  general  flight  clearance  (reference  (z)),  and  the 
ship's  blue  lighting  flight  test  aviation  facilities  certification  (reference  (aa)).  Testing  will  not  be 
conducted  under  any  conditions  without  appropriate  clearance/authorization  from  all  required 
commands. 

TEST  CONFIGURATIONS 

14.  Descriptions  of  desired  aircraft  test  configurations  are  presented  in  appendix  D,  table  II. 
As  discussed  in  appendix  D,  table  II,  the  UH-IN  and  CH-46E  will  carry  no  external  stores.  The 
UH-IN  and  CH-46E  will  carry  internal  auxiliary  fuel  tanks  to  boost  its  average  gross  weight. 
Presuming  that  appropriate  NVD  operating  clearances  are  obtained  from  CNSP,  all  aircraft  will 
conduct  NVD  operations. 

DATA  COLLECTION 

15.  Pilots  and/or  aircrew  will  employ  portable  audiocassette  tape  recorders  to  record 
comments  (via  the  aircraft  Internal  Communications  System  (ICS))  during  day  and  night 
operations. 

16.  Pilots  will  monitor  and  record  aircraft  performance  parameters  on  kneeboard  data  cards 
during  daylight  flight  test  sequences  only.  Sample  pilot  kneeboard  data  card  forms  are  presented 
in  appendix  B,  figures  6  and  7. 

1 7.  The  Shipboard  Test  Coordinator  (located  either  in  the  LSO  station,  in  HCS,  or  on  the 
bridge),  will  be  in  radio  contact  with  the  pilots  during  testing,  and  will  coordinate  general  test 
progression,  ensuring  that  the  appropriate  evolutions  are  conducted  in  accordance  with  this 
document. 
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18.  The  Bridge  Coordinator  will  ensure  that  ambient  atmospheric  conditions,  including  true 
wind,  ship  course/speed,  weather  and  sea  state  parameters  are  recorded  during  each  launch/ 
recovery  sequence.  Additionally,  he  will  calculate  and  relay  desired  relative  wind  and  ship 
course/speed  parameters  to  appropriate  ship  personnel  during  testing.  A  Bridge  Coordinator  data 
sheet  is  presented  in  appendix  B,  figure  8. 

19.  The  Aircraft  Project  Engineer,  located  in  HCS  or  the  LSO  station,  will  be  in  radio  contact 
with  the  pitots  during  testing,  and  will  record  test  conditions  (including  lighting  intensity  settings 
at  night)  and  pilot  post-event  comments.  A  sample  Aircraft  Project  Engineer  data  sheet  is 
presented  in  appendix  B,  figure  9. 

20.  The  Ship  Motion  Instrumentation  Engineer,  located  either  on  the  bridge,  in  Combat 
Information  Center  (CIC),  or  at  some  other  desirable  position  (determined  before  the  installation 
of  the  DI  ship  motion  package),  will  operate  the  DI  ship  motion  package  during  all  launch/ 
recovery  sequences.  A  sample  Ship  Motion  Instrumentation  Engineer  data  sheet  is  presented  in 
appendix  B,  figure  10. 

21 .  The  Audio  Visual  Instrumentation  Coordinator,  located  on  the  flight  deck,  on  the  hangar 
roof,  or  at  other  advantageous  locations,  will  coordinate  the  operation  of  DAVIS  (described  in 
paragraph  56b),  and  will  ensure  that  portable  VHS  format  video  camcorders  and  35mm  still 
cameras  are  employed  to  provide  both  day  and  night  documentation  of  the  video/photo 
documentation  objectives  presented  in  appendix  B,  figure  1 1 . 

METHOD  OF  TESTS 

LANDBASED  FLIGHTS 

PROFICIENCY  FLIGHTS 

22.  In  order  to  ensure  maximum  pilot  proficiency,  each  project  pilot  will  conduct  preparatory 
flights  prior  to  the  conduct  of  DI  shipboard  test  evolutions.  These  flights,  summarized  in  appendix 
D,  table  II,  will  include; 

a.  Proficiency/FCLP  flights  at  NAWCAD  Pax  River  and  the  EFP.  These  proficiency 
flights,  as  summarized  in  appendix  D,  table  II,  will  include  day  and  night  (unaided  and  aided)  pilot 
proficiency/FCLP  evolutions  at  the  EFP  (NAWCAD  Lakehurst).  The  flights  will  include 
emergency  procedures  review  (including  degraded  AFCS  waveoff  and  recovery  practice,  which 
will  be  conducted  using  the  "graceful  AFCS  degradation"  buildup  methodology  described  in 
paragraphs  42  and  43);  EFP  operations  will  also  include  procedures  familiarization  flights  as  a 
buildup  for  actual  shipboard  tests  (discussed  in  paragraphs  44  through  47). 

b.  Proficiency/FCLP  flights  at  the  providing  squadron  (if  possible)  or  other  landbased 
facility.  If  possible,  these  day/night  flights  will  be  conducted  to  include  a  review  of  local  field 
rules,  non-NVD  FCLPs,  and  emergency  procedures  review  (including  degraded  AFCS  waveoff 
and  recovery  practice,  which  will  be  conducted  using  the  "graceful  AFCS  degradation"  buildup 
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methodology  described  in  paragraphs  42  and  43).  These  flights  will  also  be  used  to  measure  land- 
based  aircraft  performance  characteristics  (in  accordance  with  paragraph  27)  for  later  comparison 
with  shipboard  aircraft  performance  characteristics. 

c.  Deck  Landing  Qualifications  (DLQs).  Each  pilot  will  meet  appropriate  DLQ 
requirements,  (presented  in  appendix  D,  table  IV)  prior  to  the  conduct  of  shipboard  DI  test 
operations.  (Note  that  although  all  pilots  flying  in  the  aircraft  during  a  DI  test  evolution  must  be  a 
NAWCAD  Pax  River  TPS  graduate/Cat  C  qualified  pilot,  all  test  aircrewmen  may  be  either  fleet 
or  Pax  River  personnel,  provided  that  they  meet  NATOPS  qualifications.  All  DLQ  sequences  will 
be  conducted  in  accordance  with  existing  NATOPS  procedures.  All  night  DLQs  will  be 
conducted  by  unaided  pilots;  NVD  evolutions  will  NOT  be  conducted.  If  atmospheric  conditions 
permit,  the  DI  general  launch/recovery  envelope  (appendix  B,  figure  12),  aligned  with  the  stern 
approach/lineup  line,  will  be  used  for  DLQ  evolutions. 

LANDBASED  HOVER  LADDER  FLIGHT  TESTS 

23.  Asa  desired  part  of  the  shipboard  launch/recovery  envelope  development  test  effort,  each 
test  aircraft  will  attempt  to  conduct  one  or  more  day  land-based  "hover  ladder"  test  sequences  at 
the  providing  squadron  or  at  some  other  land-based  facility.  These  day  tests,  summarized  in 
appendix  D,  table  V,  will  be  conducted  to  verify  predicted  gross  weight/required  torque  values  for 
the  test  aircraft,  to  quantify  both  in  ground  effect  (IGE)  and  out  of  ground  effect  (OGE)  hover 
performance  characteristics,  and  to  serve  as  a  reference  for  shipboard  hover  torque  values 
obtained  during  at-sea  testing.  As  shown  in  appendix  D,  table  II,  one  or  more  land-based  hover 
ladder  test  evolutions  will  be  conducted  with  the  actual  test  aircraft.  (Then,  as  described  in 
paragraph  29,  successive  sets  of  shipboard  hover  ladder  evolutions  will  also  be  conducted  over 
the  ship  flight  deck.)  Both  landbased  and  shipboard  evolutions  will  be  conducted  with  aircraft- 
relative  wind  speed  and  other  conditions  set  in  accordance  with  appendix  D,  table  V.  Both  land 
and  shipboard  hover  ladder  test  evolutions  will  consist  of  10-20  sec  duration  IGE  and  OGE 
daylight  hover  sequences.  As  described  in  appendix  D,  table  V,  pilots  will  conduct  each  sequence 
under  sequentially  varying  aircraft  headings  (within  +  90  deg  from  the  true  wind),  during  which 
they  will  record  the  parameters  specified  in  appendix  B,  figures  6  on  flight  data  cards. 

SHIPBOARD  FLIGHT  TESTS 

GENERAL 

24.  Shipboard  flight  testing  will  be  conducted  in  accordance  with  the  established  NAWCAD 
Pax  River  DI  test  procedures  outlined  below.  The  Shipboard  Test  Coordinator  and/or  Bridge 
engineer  will  request  the  Commanding  Qfficer  or  the  Offlcer-of-the-Deck  to  operate  applicable 
ship  systems  to  provide  the  wind-over-deck  (WOD),  ship  motion  (including  operating  ship 
rudders  to  create  artificially  increased  ship  motion,  if  desired/required),  ship  lighting,  and/or  other 
appropriate  conditions  required  for  each  test  sequence.  After  the  ship  has  attained  the  desired 
conditions,  one  or  more  test  evolutions  will  be  conducted  by  the  aircraft  as  necessary.  During 
each  flight  test  evolution,  at  least  one  NAWCAD  Pax  River  Aircraft  Project  Engineer  will  have 
direct  aircraft  radio  communication  capability.  Except  for  the  approved  variations  specified  in  this 
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test  plan  (paragraphs  16  and  42-49),  all  shipboard  sequences  will  employ  NATOPS  approach, 
recovery,  and  departure  procedures. 

SHIPBOARD  HOVER  LADDER  FLIGHT  TESTS 

25.  As  part  of  the  shipboard  launch/recovery  envelope  development  test  effort,  one  or  more 
day  shipboard  "hover  ladder"  test  sequences  will  be  conducted.  These  daylight  tests  will  be 
conducted  to  compare  gross  weight  and  required  torque  values  with  NATOPS  predictions,  and 
with  land-based  values.  Also,  the  test  data  will  be  used  to  quantify  IGE,  OGE,  and  single  engine 
aircraft  hover  performance  characteristics.  (Note:  NO  single  engine  flights  will  be  attempted 
during  any  at-sea  testing;  any  single  engine  data  resulting  from  this  test  will  be  based  on  dual 
engine  extrapolations  only.)  As  depicted  in  appendix  D,  table  II,  shipboard  hover  ladder  test 
evolutions  will  be  conducted  over  the  deck  prior  to  and/or  in  conjunction  with,  planned 
launch/recovery  flight  test  operations.  To  provide  the  best  comparison  with  land-based  evolutions, 
the  shipboard  hover  ladder  evolutions  will  be  conducted  with  WOD  speed  and  ship  motion 
stabilized  to  the  greatest  extent  practicable  Shipboard  hover  ladder  evolutions  will  consist  of  10- 
20  sec  duration  IGE  and  OGE  daylight  hover  sequences,  conducted  under  the  conditions  specified 
in  appendix  D,  table  V.  As  described  in  appendix  D,  table  V,  each  sequence  will  be  conducted 
under  sequentially  varying  aircraft  headings  (within  +  90  deg  relative  to  the  ship's  bow),  during 
which  pilots  will  record  the  parameters  specified  in  appendix  B,  figures  6  on  flight  data  cards. 

LAUNCH/RECOVERY  FLIGHT  TEST  OPERATIONS 

General 

26.  The  day/night  flight  test  methodologies  discussed  below  were  selected  in  order  to 
maximize  the  amount  and  validity  of  test  results  for  a  variety  of  evaluations  conducted  within  a 
limited  time  frame.  In  accordance  with  appendix  D,  table  I  and  II,  the  majority  of  day  and  night 
test  operations  are  scheduled  to  consist  of  multiple  shipboard  approach/recovery/launch/departure 
evolutions.  However,  as  shown  in  appendix  D,  table  II,  various  other  component  tests  may  also 
be  conducted  while  underway,  depending  on  ambient  conditions.  In  accordance  with  appendix  D, 
table  I  and  II,  during  night  operations,  multiple  shipboard  approach/recovery/launch/  departure 
evolutions  (NVD  and  non-NVD)  will  be  conducted  under  NVD  compatible  blue  lighting  within 
envelopes  developed  during  daylight  conditions. 

27.  Pilots  will  conduct  shipboard  launch/recovery  evolutions  under  day  and  night  conditions, 
to  limits  dictated  by  pilot  ratings  or  by  ambient  wind  or  sea  state  test  conditions.  Pilots  will  assign 
ratings  of  each  approach/recovery/launch/departure  evolution  in  accordance  with  guidelines  set 
forth  in  the  DI  Pilot  Rating  Scale  (PRS),  Turbulence  Rating  Scale,  Handling  Qualities  Rating 
Scale,  and  Vibration  Rating  Scale  (which  are  presented  in  appendix  C,  scales  I  through  IV, 
respectively)  as  applicable.  Pilot  ratings  will  reflect  pilot  workload  and  performance  during 
shipboard  launch/recovery  tasks;  Lfnacceptable  pilot  ratings  are  typically  caused  by  degradation  of 
flying  qualities  and/or  performance  in  the  shipboard  environment. 
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28.  Specific  limitations  that  might  be  encountered  during  testing  include  insufficient  flight 
control,  torque,  rotor  speed,  or  power  margins,  excessive  pilot  workload,  turbulence  or  ship 
motion,  inadequate  visual  cues,  or  inadequate  clearance  distance  between  aircraft  and  ship 
structure.  A  summary  of  critical  potential  flight  limitations  (and  minimum  ratings  that  will  be 
assigned  to  those  evolutions,  if  reached)  is  presented  in  appendix  D,  table  VII. 

Day  Operations 

General 

29.  The  major  phase  of  shipboard  testing  will  be  conducted  during  daylight,  under  elevated 
sea  state  conditions  as  practicable.  During  this  phase,  the  test  team  will  attempt  to  produce  a 
variety  of  wind  and  ship  motion  conditions  by  systematically  varying  the  ship's  course  and  speed 
for  a  given  true  wind/true  wave  direction  condition,  in  turn  facilitating  investigation  of  the  effects 
of  previously  unexplored  relative  wind  speed  and  direction  and/or  ship  motion  magnitudes  on  a 
pilot's  ability  to  conduct  shipboard  launch/recovery  operations.  After  each  systematic  variation  of 
relative  wind  speed  and  direction,  pilots  will  conduct  multiple  repeated  shipboard 
approach/recovery/launch/  departure  sequences.  In  order  to  ensure  a  safe  buildup  for  each  specific 
(wind  and  motion)  test  condition,  all  day  tests  will  adhere  to  the  accepted  DI  buildup 
methodology  outlined  in  paragraphs  34  through  41,  as  determined  by  test  progression,  test 
priority,  atmospheric  conditions,  or  other  emergent  requirements.  These  tests  will  include  both 
bow  and  beam  wind  envelope  explorations;  after  completion  of  the  bow  wind  velocity  buildup  test 
procedures,  the  test  team  will  investigate  the  effects  of  WOD  direction  variation  on  shipboard 
approach/recovery/launch/departure  sequences.  Both  phases  will  be  described  in  the  following 
paragraphs. 

Day  Bow  Wind  Envelope  Development 

30.  The  initial  WOD  speed/direction  that  will  be  tested  for  each  aircraft  will  be  located  within 
the  day  general  DI  launch/recovery  envelope  (aligned  with  the  up-the-stern  lineup  line)  presented 
in  appendix  B,  figure  12).  These  initial  WOD  conditions  will  be  chosen  so  as  to  minimize  ship 
motion  to  the  greatest  extent  possible  for  the  given  set  of  sea  state  and  weather  conditions.  Under 
these  conditions,  pilots  will  conduct  one  or  more  stern  approach/recovery/launch/  departure 
sequences.  After  completion  of  each  desired  approach/recovery/launch/  departure  sequence,  pilots 
will  assign  ratings  and  other  applicable  comments  to  the  sequence,  and  radio  them  back  to 
engineers  aboard  ship.  For  each  ship  course/speed  (or  equivalently,  WOD  speed/direction) 
combination  selected,  each  aircraft  will  attempt  at  least  one  recovery/  launch  sequence;  pilots  may 
also  conduct  additional  recoveries,  however,  if  warranted  by  emergent  conditions. 

3 1  After  completion  of  all  desired  approach/recovery/launch/departure  sequences  attempted 
under  a  given  set  of  ship  course/speed  conditions,  the  ship  will  be  maneuvered  to  produce  new 
relative  WOD  (and  resultant  ship  motion)  conditions  for  the  next  set  of  sequences.  For  each 
successive  ship  maneuver  conducted  during  this  initial  phase  of  testing,  the  WOD  speed  will  be 
progressively  increased  (at  approximately  constant  relative  direction,  down  the  bow)  in 
approximately  5  kt  increments  to  the  maximum  attainable  value,  or  until  an  unacceptable  (PRS-3 
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or  PRS-4)  rating  is  assigned  to  an  approach/recovery/launch/departure  sequence.  If  a  pilot  assigns 
a  PRS-3  to  any  sequence,  the  sequence  may  be  repeated  for  verification  only  with  pilot  and  test 
coordinator's  concurrence.  After  verification  of  such  PRS-3  sequences,  subsequent  bow  wind 
approach/recovery/launch/departure  sequences  will  only  be  conducted  at  relative  WOD  that  are 
reduced  in  magnitude  by  at  least  5  kt.  If  a  pilot  assigns  a  PRS-4  to  any  sequence,  the  relative 
WOD  speed  and  ship  motion  will  be  reduced  to  levels  corresponding  to  a  previous  PRS-2  or 
PRS-1  rating  before  attempting  another  sequence.  Sequences  conducted  under  conditions  that 
produced  PRS-4  ratings  will  not  be  repeated;  any  PRS-1,  PRS-2,  or  PRS-3  sequence  may  be 
repeated  as  required/desired,  with  pilot  and  test  coordinator  concurrence. 

Day  Beam  Wind  Envelope  Development 

32.  During  this  phase  of  testing,  the  ship  will  be  maneuvered  progressively  so  as  to  vary  WOD 
direction  to  port  or  starboard  in  approximately  1 5  deg  increments,  while  keeping  the  relative 
WOD  speed  constant,  or  at  the  maximum  WOD  speed  attainable.  Pilots  will  conduct  one  or  more 
day  approach/recoveiy/launch/departure  sequences,  using  stem  approach,  for  each  ship 
course/speed  combination  selected. 

33.  After  completion  of  each  beam  wind  recovery  sequence,  pilots  will  radio  PRS  ratings  and 
applicable  comments  back  to  the  test  engineers  aboard  ship.  Again,  if  a  pilot  assigns  a  PRS-3  to 
any  sequence,  the  sequence  will  be  repeated  for  verification  only  with  pilot  and  test  coordinator's 
concurrence.  If  a  verifiable  PRS-3  is  assigned  to  an  approach/recovery/launch/  departure  sequence 
conducted  during  this  phase  of  testing,  the  relative  WOD  speed  will  be  reduced  in  (approximately) 
a  5  kt  increment  (while  maintaining  a  constant  relative  WOD  direction)  before  conducting  another 
sequence.  If  several  verifiable  PRS-3  ratings  are  assigned  to  successively  lower  WOD  speed 
conditions  along  a  given  azimuth,  sequences  will  be  conducted  at  still  lower  WOD  speeds,  until  a 
WOD  speed  condition  producing  a  satisfactory  PRS  rating  is  attained.  If  a  pilot  assigns  a  PRS-4 
to  any  sequence,  the  relative  WOD  speed,  direction,  and  ship  motion  will  be  reduced  to  levels 
corresponding  to  a  previous  PRS-2  or  PRS-1  rating  before  another  sequence  is  attempted.  Again, 
during  this  phase  of  testing,  WOD/motion  conditions  that  produced  PRS-4  ratings  will  not  be  re¬ 
evaluated;  any  PRS-1,  PRS-2,  or  PRS-3  sequence  may  be  repeated  as  required/desired  only  with 
pilot  and  shipboard  test  coordinator  concurrence. 

Miscellaneous  Day  Envelope  Development  Test  Procedures 

34.  For  each  aircraft,  initial  WOD  conditions  for  each  successive  day  shipboard  flight  test 
envelope  development  period  will  be  located  within  previously  tested  envelope  boundaries.  Also, 
within  the  constraints  allowed  by  ambient  winds  and  previous  test  results,  the  initial  WOD 
conditions  on  each  successive  test  period  will  be  chosen  so  as  to  provide  both  a  rating  verification 
check  and  to  provide  torque  data  for  post-test  nonstandard  day  performance  estimation. 

(Basically,  during  the  first  test  period  for  each  aircraft,  and  at  other  times  as  required,  the  aircraft 
will  refly  one  or  more  previously  tested  WOD  conditions  that  corresponded  to  PRS-1  and/or 
PRS-2  ratings.  These  reflown  points  will  thus  be  used  to  verify  pilot  ratings  of  tasks  conducted  in 
previous  days,  in  an  attempt  to  improve  rating  constancy  and  quality  of  data.  Data  from  these 
points  will  then  also  be  used  to  estimate  aircraft  performance  for  more  extreme  GW  and/or 
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atmospheric  conditions).  Previously  tested  WOD  conditions  for  each  aircraft  may  also  be  retested 
to  investigate  the  effects  of  aircraft  loading,  ship  motion,  or  weather  conditions,  throughout  the 
tests  as  required  by  each  day's  data  analysis. 

35.  In  general,  pilots  will  relay  comments  and  PRS  ratings  only  after  completion  of  each 
approach/recovery/launch/departure  sequence;  however,  pilots  may  also  elect  to  relay 
unsatisfactory  (PRS-3  or  PRS-4)  approach/recovery  PRS  ratings  while  the  aircraft  is  on  deck, 
prior  to  launch.  In  this  event,  pilot  and  engineers  will  verbally  confirm/establish  the  desired 
launch/departure  WOD  and/or  resultant  ship  motion  conditions  (subject  to  all  limiting  PRS 
procedures  discussed  in  this  paragraph)  before  launch. 

36.  The  procedures  outlined  in  paragraphs  33  through  40  will  be  used  to  develop  day 
shipboard  operational  launch/recovery  wind  and/or  ship  motion  envelopes  for  each  aircraft 
throughout  the  embarked  test  period.  Testing  will  continue  until  the  maximum  safe  day 
launch/recovery  envelopes  are  developed,  within  constraints  posed  by  available  test  time. 

Degraded  Flight  Control  Systems  Recovery  Tests 

37.  Testing  to  examine  the  effects  of  degraded  AFCS  parameters  on  CH-46E,  and  of 
degraded  SCAS  parameters  on  UH-IN  shipboard  recovery  capability  may  be  conducted  during 
daylight  test  periods,  if  test  time  and/or  ambient  conditions  permit,  and  if  degraded  AFCS 
recoveries  were  conducted  during  buildup  evolutions  at  Pax  River  and/or  the  EFP.  All  shipboard 
day  degraded  AFCS  recoveries  will  be  conducted  at  WOD  and  ship  motion  conditions  located 
within  previously  tested  day  nondegraded  AFCS  launch/recovery  envelopes.  No  night  degraded 
AFCS  recoveries  will  intentionally  be  attempted. 

38.  Degraded  AFCS  testing  will  employ  a  "graceful  degradation"  buildup  sequence,  in  which 
the  effects  of  lesser  degradations  are  investigated  before  proceeding  to  more  severe  ones.  During 
initial  degraded  AFCS  test  recoveries,  the  pilot  will  artificially  degrade  the  aircraft's  flight  control 
systems  by  selecting  the  appropriate  degradation  (AFCS  RELEASE  (SAS  1,  SAS2,  and 
AUTOPILOT  OFF)  for  the  CH-46E,  and  SCAS  OFF  for  the  UH-IN)  prior  to  commencing  an 
approach  to  foe  ship.  Unless  emergency  recoveiy  is  required,  the  AFCS  will  remain  disengaged 
for  the  duration  of  the  approach/recovery  task.  After  recovery  to  the  flight  deck,  the  AFCS  will  be 
reengaged  prior  to  subsequent  launch  and  climbout.  This  procedure  may  then  be  repeated,  at  pilot' 
s  discretion,  with  a  second  and  final  system  degradation  (SAS/BOOST  OFF  for  the  CH-46E,  and 
SCAS^OOST  OFF  for  the  UH-IN)  for  another  set  of  recoveries  under  the  same  WOD 
conditions.  Each  multiply-degraded  AFCS  mode  recovery,  however,  will  only  be  conducted  for 
those  WOD  conditions  previously  tested  as  safe  for  single  degraded  mode  AFCS  recoveries.  After 
completion  of  all  desired  degraded  AFCS  recoveries  at  a  given  WOD,  the  ship  will  be 
maneuvered,  producing  a  different  WOD  condition,  at  which  another  set  of  degraded  recoveries 
will  be  conducted.  Degraded  AFCS  PRS  ratings  will  be  assigned  using  the  same  criteria  as  for 
nondegraded  recoveries. 
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AERIAL  PHOTOGRAPHY  SEQUENCES 

39.  As  discussed  in  paragraph  24,  DI  test  engineers  and  test  aircrew  will  document  shipboard 
DI  test  evolutions  with  both  video  and  still  cameras.  Provided  test  time,  test  priorities,  and 
ambient  conditions  permit,  the  test  aircrew  will  conduct  aerial  documentation  of  selected 
day/night  test  sequences.  In  accordance  with  appendix  D,  table  II,  during  such  aerial  photographic 
sequences,  the  aircraft  will  conduct  multiple  approach/hover/waveofF  evolutions  (no  full  recovery 
photography  evolutions  will  be  conducted).  All  aerial  photo  documentation  sequences  will  be 
conducted  under  previously  tested  WOD,  ship  motion,  and  deck  lighting  conditions;  no  aerial 
photo  documentation  sequences  will  be  conducted  during  envelope  development  test  operations. 

Night  Operations 

General 

40.  Night  launch/recovery  test  operations  will  be  conducted  under  blue  flight  deck  lighting. 
Night  launch/recovery  operations  may  include  the  evaluations  listed  in  appendix  D,  table  II, 
dependent  on  ambient  conditions.  Night  launch/recovery  test  methods  and  procedures  will  be 
generally  similar  to  day  test  methods  and  procedures,  although  self-imposed  night  shipboard  flight 
test  safety  precautions  will  mandate  some  test  procedural  modifications.  First,  pilots  will  NOT  be 
required  to  record  test  data  on  pilot  data  cards  during  night  test  operations  (instead,  either  they  or 
the  aircrewmen  will  have  portable  tape  recorders  to  record  comments).  Secondly,  tape  measure 
directional  control  (pedal  position)  instrumentation  (described  in  paragraph  54)  will  NOT  be  used 
during  any  night  operations.  Finally,  since  these  safety  precautions  will  limit  the  amount  of 
quantitative  data  obtained  during  night  test  operations,  all  night  approach/recovery/ 
launch/departure  sequences  will  only  be  conducted  under  those  WOD  and  ship  motion  conditions 
previously  established  as  safe  during  day  test  operations.  Thus,  night  flight  test  sequences  will 
NOT  be  attempted  under  any  WOD/ship  motion  conditions  not  contained  within  a  tested  day 
envelope.  However,  during  night  testing,  each  pilot  will  NOT  necessarily  be  restricted  to 
conducting  test  evolutions  only  under  those  WOD/ship  motion  conditions  he  personally  evaluated 
in  daylight.  Additional  safety  precautions  are  summarized  in  paragraph  69. 

Flight  Deck  Lighting  Compatibility  Evaluation 

4 1 .  Current  test  plans  call  for  pilots  to  conduct  all  shipboard  and  EFP  night  operations  under 

NVD  compatible  blue  flight  deck  lighting.  (This  is  not  required  for  the  test;  it  is  only  desired.) 

When  conditions  are  favorable  for  night  shipboard  flight  testing,  pilots  will  informally  evaluate  the 
adequacy  of  the  blue  lighting  concurrently  with  test  operations.  When  ambient  conditions  are  too 
calm  to  allow  the  conduct  of  launch/recovery  envelope  expansion  tests,  pilots  will  conduct  a  night 
NVD  and  non-NVD  lighting  compatibility  evaluation  in  accordance  with  the  procedures  described 
below.  The  test  evolution  summary  provided  in  appendix  D,  table  VIII  will  govern  the  conduct  of 
this  evaluation.  As  specified  in  the  tables,  during  this  evaluation,  pilots  will  conduct  multiple 
approach/recovery/takeoff/departure  sequences,  each  under  various  combinations  of  flight  deck 
lighting  component  intensity,  and  recovery  type.  During  and  after  each  evolution,  pilots,  aircrew. 
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Helicopter  Control  Officers  (HCOs),  LSEs,  and  other  ship  personnel  will  be  asked  to  describe, 
critique,  and  comment  on  the  visual  compatibility  (pilots  will  radio  all  comments  while  on  deck). 

42.  During  all  lighting  evaluation  testing,  the  ship  HCO  will  modulate  the  intensity  of  required 
shipboard  VLA  components  prior  to  the  commencement  of  an  approach  to  the  ship.  The  intensity 
of  any  modulated  VLA  system(s)  will  remain  fixed  throughout  the  duration  of  the 
approach/recovery/launch/departure  task,  unless  otherwise  directed  by  the  pilots;  modulated  VLA 
components  will  be  varied  at  any  point  in  a  test  sequence  if  so  requested  by  the  pilot.  For  each 
sequence,  pilots  will  assign  PRS  ratings  the  same  as  for  a  nondegraded  landing.  During  each 
lighting  test  period,  the  ship  will  attempt  to  maintain  a  constant  course  and  speed  for  as  many 
consecutive  evolutions  as  possible,  thus  minimizing  adverse  effects  caused  by  variations  in  relative 
wind  speed  and  direction.  Lighting  evaluation  testing  will  immediately  cease,  at  pilot/engineer 
discretion,  if  atmospheric  conditions  warrant. 

SHIPBOARD  NON-FLIGHT  TESTS 

SHIPBOARD  COMPATIBILITY/AVIATION  FACILITIES  ADEQUACY  EVALUATION 

43.  In  accordance  with  appendix  D,  table  IX,  evaluation  of  the  adequacy  of  ship  aviation 
facilities,  including  the  prototype  flight  deck  lighting  package  and  hangar/HCS/LSO  station  layout 
will  be  conducted  concurrently  with  each  of  the  test  areas  outlined  above.  In  addition,  the 
shipboard  compatibility  and  adequacy  of  the  test  aircraft  will  be  examined  concurrently  with  the 
DI  tests.  Refueling  is  planned  to  occur  approximately  every  two  hours;  although  not  specifically 
planned,  underway  periods  will  also  include  significant  deck  handling  tasks,  such  as  aircraft 
tiedown,  aircraft  traversing,  and  aircraft  hangaring.  Each  such  sequence  that  does  occur  will  be 
qualitatively  evaluated  by  pilots,  flight  deck  crew,  engineers,  and  shipboard  observers,  and 
documented  by  shipboard  photographers,  to  assess  UH-IN  and  CH-46E  shipboard  operational 
compatibility  with  RAST-equipped  FFG  7  class  ships. 

INSTRUMENTATION 

AIRCRAFT-MOUNTED  INSTRUMENTATION 

44.  Dependent  on  operational  test  considerations,  tape  measure  devices  may  be  mounted  to 
the  directional  control  pedals  to  measure  control  excursions  experienced  during  testing.  Portable 
audio  cassette  tape  recorder  devices  will  also  be  employed  to  record  pilot/aircrew  comments  via 
the  aircraft  Internal  Communications  System  (ICS).  Tape  recorder  and  measuring  tape  mounting 
locations  and  procedures  will  be  in  accordance  with  standard  USN  Test  Pilot  School  (USNTPS) 
practice.  During  aerial  documentation  sequences,  aircrew  will  embark  the  aircraft  with  portable, 
hand-held  video  and  still  cameras;  these  will  be  connected  into  the  aircraft  ICS  to  record  pilot 
comments  directly  onto  the  video.  Other  than  the  items  mentioned  here,  no  other  test 
instrumentation  will  be  mounted  in  the  test  aircraft. 
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SHIPBOARD  INSTRUMENTATION 

45.  Test  instnimentation  and  support  equipment  external  to  the  aircraft  is  scheduled  to  include 
the  following: 

a.  DI  Audio  Visual  Instrumentation  System  (DAVIS),  used  to  document  flight  test 
evolutions.  DAVIS  consists  of  two  or  more  CCTV  cameras  hooked  into  a  quad-split  monitor, 
whose  output  is  then  mixed  with  engineer  and  aircrew  radio  comments  (acquired  by  a  portable 
scanner  unit)  and  then  fed  into  a  VCR. 

b.  DI  Ship  Motion  Package  (SMP),  used  to  document  ship  motion.  The  SMP,  which 
includes  COMPAQ  386  portable  computer,  ship  motion  instrumentation  box,  assorted  cables,  and 
SMP  data  collection/analysis  software,  possesses  the  capability  to  measure  and  record 
displacements,  rates,  and  accelerations  for  ship  roll,  pitch,  and  yaw  motions  at  3  Hz.  The  SMP 
will  also  record  ship  course/speed  and  ship  anemometer  speed/direction  inputs. 

c.  Portable  camera  equipment,  (35mm  still  and  VHS  video)  used  by  NAWCAD  Pax  River 
personnel  aboard  ship  and  inside  the  test  aircraft  to  document  test  operations.  Any  photography 
conducted  from  within  the  aircraft  may  occur  during  launch/recovery  evolutions,  but  will  not 
occur  during  any  untested  wind/ship  motion/ship  lighting  conditions. 

d.  Portable  hand-held  wind  anemometers,  used  to  document  shipboard  airwake/  airflow 
charactenshcs.  During  periods  in  which  the  aircraft  is  not  flying,  NAWCAD  Pax  River  personnel 
will  take  wind  speed/direction  readings  at  various  flight  deck  locations. 


e.  Portable  walkie-talkie  and  cellular  telephone  units,  used  for  shipboard  test  team 
internal/external  communications. 

f  Commercially  available  soap  bubble  solution  (modified  to  improve  bubble  wall  strength 
for  high  wind  applications),  used  in  conjunction  with  hand-held  anemometers  and  video  cameras 
to  visualize  and  document  flight  deck  airflow  patterns  during  flight  and  non-flight  operations. 

g.  Low  light  level  NVD  video  and  photometry  equipment  will  be  used  to  document  night 
test  operations  and  illumination  levels. 

h.  Portable  audiocassette  tape  recorder  devices  will  be  used  aboard  ship  to  record  test 
team  comments  via  the  ICS  during  testing,  and  during  post-test  team  debriefs. 

SUPPORT  REQUIREMENTS 

GENERAL 


j  The  detachment  OinC  and  shipboard  test  coordinator  conducted  prevail  conferences 
aboard  USS  INGRAHAM  on  Februaiy  25,  1995.  Detailed  shipboard  and  aircraft  support 
requirements  are  also  addressed  in  the  Operations  Plan  (reference  (ft)),  a  preliminaiy  version  of 


43 


which  will  be  presented  to  shipboard  personnel  prior  to  testing.  Additional  shipboard  briefings, 
which  will  also  review  support  requirements,  will  be  conducted  after  initial  shipboard  arrival,  prior 
to  the  conduct  of  flight  operations. 

47.  Appendix  D,  table  III  provides  a  tentative  shipboard  test  schedule.  All  test  team  members 
are  scheduled  to  walk  aboard  ship  while  it  is  pierside.  Test  aircrews  may  embark/debark  the  ship 
while  underway,  via  the  test  aircraft.  After  conclusion  of  NATO?  S -requisite  pilot/shipboard 
personnel  briefings,  NAWCAD  Pax  River  pilots  will  conduct  DLQs  and  test  operations  as  called 
out  in  this  test  plan.  The  aircraft,  aircrews,  and  civilian  test  team  members  are  scheduled  to  remain 
aboard  ship  each  underway  night;  all  detachment  personnel  are  tentatively  scheduled  to  stay 
aboard  ship  for  the  duration  of  its  underway  period.  Pilots/  aircrew  may  disembark  the  ship  early, 
prior  to  port  arrival.  DI  engineers  are  scheduled  to  walk  off  the  ship  after  it  returns  to  port.  If 
tests  are  completed  early,  or  if  the  ship's  at-sea  schedule  changes,  all  test  team  members  may  fly 
to/from  the  ship  while  it  is  at  sea,  but  only  with  appropriate  permission  from  COMNAVAIRPAC. 

FLEET  SUPPORT 

48.  COMNAVSURFPAC  (N42)  will  coordinated  to  provide  requisite  NVD  training  to  the 
ship's  crew  to  allow  conduct  of  low  level  NVD  operations. 

49.  USS  INGRAHAM  will  host  the  embarked  test  detachment  and  will  also  provide  primary 
SAR  and  refueling  support.  If  required,  USS  INGRAHAM  will  also  provide  aircraft  hangaring 
and  maintenance  facilities  support. 

50.  Marine  Light  Attack  Helicopter  Squadron  (HMLA),  from  MCAS  Camp  Pendleton,  CA, 
will  provide  the  embarked  UH-IN  test  aircraft  and  maintenance/spares  support. 

51.  Marine  Medium  Helicopter  Squadron  (HMM),  from  MCAS  Tustin,  CA,  will  provide  the 
embarked  CH-46E  test  aircraft  and  maintenance/spares  support. 

NAVAIRSYSCOM  SUPPORT 

52.  NAVAIRSYSCOM  has  tasked  and  funded  the  conduct  of  this  test.  NAVAIRSYSCOM 
(PMA  251  and/or  AIR  530)  and  NAWCAD  Lakehurst  will  also  coordinate  with  NAWCAD  Pax 
River  (RW40)  to  provide  flight  clearances  and  deck  certifications  to  enable  the  conduct  of 
shipboard  flight  tests. 

NAWCAD  LAKEHURST  SUPPORT 

53.  NAWCAD  Lakehurst  will  make  available  and  operate  the  Elevated  Fixed  Platform 
available  during  day/night  NVD/non-NVD  launch/recovery  practice  evolutions  prior  to  shipboard 
testing.  If  available,  NAWCAD  Lakehurst  will  coordinate  with  COMNAVSURFPAC  to  provide 
NVD  compatible  lighting  filters  and  lamps. 
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NAWCAD  PAX  RIVER  SUPPORT 


54.  NAWCAD  Pax  River  will  provide  UH-1 N  and  CH-46E  test  pilots  (RW40  and  RW60), 
aircrewmen  (Dyncorp  and  RW60),  DI  flight  test  engineers  (RW40),  and  all  associated  test 
support  instrumentation  and  equipment. 

55,  In  accordance  with  OPNAVINST  3750,6Q,  NAWCAD  Pax  River  MD  will  assume  all 
incident  reporting  accountability/responsibility  whenever  NAWCAD  Pax  River  pilots  act  as  HAC 
in  fleet  aircraft. 


56.  NAWCAD  Pax  River  (RW40)  will  coordinate  to  ensure  that  the  NVD  compatible  light 
filters  and  lamps  arrive  safely  at  the  test  ship,  and  will  install/remove  all  light  filters  and  lamps  for 
the  test. 


SPECIAL  PRECAUTIONS 

57^  A  NAWCAD  Pax  River  safety  checklist  is  included  as  appendix  E  to  ensure  that  adequate 
safety  precautions  are  followed.  In  addition,  the  following  special  precautions  will  be  followed: 

a.  All  project  flights  will  be  conducted  with  the  minimum  aircrew  required  for  the  safe  and 
efficient  conduct  of  operations. 

b.  Day  and  night  (NVD  and  non-NVD)  FCLP  evolutions  (either  at  the  EFP  or  at 
NAWC^  Pax  River)  will  be  conducted  to  ensure  pilot  proficiency.  FCLP  operations  will  include 
day/night  degraded  AFCS  recovery  sequences  as  a  buildup  and  prerequisite  for  shipboard  test 
operations.  Degraded  mode  AFCS  waveoffand  simulated  engine  failure  waveoff  procedures  will 

a  so  be  practiced  during  FCLPs.  The  techniques  employed  during  degraded  mode  AFCS  practice 
and  during  shipboard  degraded  AFCS  testing  are  identical,  and  are  summarized  in  paragraphs  42 


c.  After  shipboard  arrival  of  the  aircraft,  and  prior  to  the  start  of  actual  tests,  the  test  team 
will  conduct  a  test  procedural  briefing  with  the  flight  deck  crew.  Topics  of  the  brief  will  include 
aircrew  rescue  procedures  for  the  event  of  flight  deck  crash  or  fire,  aircraft  tie  down,  power-up 
retuel/defuel,  and  deck  movement  procedures.  Review/summaiy  of  all  shipboard  air  control  ’ 

procedures,  deck  status  light/recoveiy  signals,  and  VLA  component  names  will  also  be  conducted 
pnor  to  flight  operations. 

d.  Test  objectives  include  night  flights,  without  NVD,  under  ambient  illumination  levels 
providing  less  than  0.0022  lux;  a  summary  of  predicted  operational  area  clear  sky  ambient 
Illumination  levels  (determined  using  the  USMC  Light  Level  Planning  Calendar  computer 
program)  is  presented  in  appendix  B,  figure  15. 

A  j  j  (unaided)  degraded  AFCS  shipboard  test  operations  will  not  be  conducted  (day 
degraded  mode  operations  may  be  conducted,  but  are  themselves  a  low  priority.) 
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f.  Each  night  test  evolution  will  be  conducted  under  WOD/ship  motion  conditions 
previously  tested  as  safe  during  daylight.  Deck  lighting  conditions  employed  during  each  night 
evolution  will  be  adjusted  according  to  a  test  matrix  and/or  to  pilot  preference. 

g.  A  pilot  will  only  conduct  testing  On  any  given  night  provided  that  he  has  previously 
completed  both  day  and  night  DLQs.h.  Pilots  and  aircrew  will  wear  appropriate  anti-exposure 
suits  during  both  mandatory  and  "optional-wear"  water/air  temperature  conditions. 

MANAGEMENT 

FUNDING  REQUIREMENTS 

58.  Funding  to  cover  DI  test  costs  has  been  provided  by  PMA  251,  under  reference  (g).  These 
funds  expire  on  30  Sep  95.  The  cost  estimate  for  this  DI  test  is  presented  below  in  table  I. 

Table  I 

UH-IN,  CH-46E/FFG  7  DI  Test  Costs 


Effort/Item 

Cost  Center 

Amount 

DI  Labor,  Report  Writing 

(RWJO) 

$  55.0  K 

Travel/Lodging/Per  Diem  Costs 

(RWJO,RWDO) 

$  20.0  K 

NAWCAD  Prof  Flight/Fuel  Costs 

(RWDO) 

$  30.0  K 

Test  Flight/Fuel  Costs 

(N/A) 

$  5.0  K 

NAWCAD  Lakehurst  EFP  Costs 

(N/A) 

$  8.0  K 

Material  Costs/Photo/Misc. 

(RWJO/AOPO) 

$  6.0  K 

Report  Printing  Costs 

(N/A) 

$  6.0  K 

TOTAL  DI  TEST  COSTS 

$130.0  K 

SCHEDULE/MILESTONES 

59.  A  schedule/milestone  chart  is  presented  on  the  test  plan  cover. 

PERSONNEL  ASSIGNMENT 

60.  The  NAWCAD  Pax  River  DI  test  team  members  are  listed  in  appendix  D,  table  X. 
PROJECT  SECURITY 

6 1 .  What  is  the  overall  security  classification  of  the  project? 

Response:  This  is  an  UNCLASSIFIED  project  and  test.  The  data  requiring  protection 
under  this  plan  is  sensitive  rather  than  classified.  Sensitive  data  is  unclassified  information  that, 
when  accumulated,  may  allow  a  Foreign  Intelligence  Service  (FIS)  to  analyze  it  by  compilation  or 


46 


mosaic  methods  to  derive  classified  information.  For  this  DI  test  program,  sensitive  data  will  not 
be  transmitted  via  unencrypted  (clear)  telemetry  data  links  and/or  unencrypted  voice 
communications. 

A.  Examples  of  sensitive  data  include  the  following; 

1  System  performance  information  and  comments  on  the  success  or  failure  oftesting. 

2.  Technical  performance  specifications  of  the  system  and  its  performance  capabilities. 

3.  Performance  and  evaluation  of  the  system  undergoing  test  and  evaluation. 

4.  Formal  test  results  indicating  system  limitations,  deficiencies,  and  corrective  actions. 

5.  Details  of  test  methodology,  including  locations  and  dates  and  times  oftesting. 

B.  Examples  of  sensitive  material  include,  but  are  not  limited  to,  the  following: 

1.  Technical  Reports 

2.  Formal  BIS  and  Yellow  Sheet  Reports 

3.  Formal  briefings  which  contain  the  types  of  material  listed  above. 

4.  Information  indicating  performance  of  hardware  listed  in  the  H-60  series  Security  Classification 
Guides. 


Sensitive  data  will  be  marked  and  handled  as  "FOR  OFFICIAL  USE  ONLY"  (FOUO) 
data,  consistent  with  the  guidance  of  SECNAVINST  5720.42D.  At  no  time  will  FOUO 
information  be  exposed  to  unauthorized  personnel.  Dynamic  Interface  test  sensitive  data  shall  be 
stored  in  locked  receptacles  such  as  file  cabinets,  desks,  or  bookcases. 

The  FOUO  marking  will  not  be  used  on  technical  documents  which  require  a  distribution 
statement  pursuant  to  DoD  Directive  5230.24.  Since  no  other  means  of  protection  is  identified  for 
protecting  technical  documents  containing  sensitive  data,  the  documents  must  be  handled  and 
stored  as  described  above. 

This  test,  or  portions  thereof,  will  be  conducted  at  a  host  facility  or  vessel  that  is  not 
located  on  Patuxent  River,  Maryland,  therefore  the  project  engineer  and  test  personnel  will  mark, 
control,  transmit  and/or  publish  all  For  Official  Use  Only  information  in  accordance  with 
NAVAIRTESTCEN  Instruction  3070.3.  All  information  regarding  this  test  will  be  marked, 
processed,  stored,  and  published  and/or  disseminated  as  required  by  appropriate  DoN  published 
policy  and  guidance. 

All  electrically  recorded  media,  e.g.,  digital  or  analog  tapes,  photographs,  charts,  graphs, 
drawings,  slides,  and  video  tapes  will  be  reviewed  for  security  classification  and  marked  in 
accordance  with  special  instructions,  security  classification  guidance,  or  at  a  minimum  in 
accordance  with  OPNAVINST  5510.  IH. 

A  definitive  OPSEC  annex  to  this  test  plan  is  not  required  and  this  action  has  been 
coordinated  with  RWATD  OPSEC  coordinator  and  approved.  Host  facility  OPSEC  procedures 
may  be  incorporated  into  this  test  plan. 
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REPORTS 


POST  FLIGHT  REPORTS 

62.  Each  pilot  participating  in  a  flight  test  period  will  complete  and  submit  a  post-flight  report 
(daily)  to  the  DI  aircraft  project  engineer  within  24  hours  of  completion  of  the  test  period. 

PROJECT  REPORTS 

63.  All  post-test  project  reports  will  be  co-authored  by  the  project  team.  NAWCAD  Pax  River 
shall  submit  the  following  reports/briefs  to  the  appropriate  organizations: 

a.  A  Message  Report  (MR),  that  summarizes  the  extent  of  testing  and  results,  will  start 
RW  routing  within  10  working  days  after  the  project  engineering  team  returns  from  the  test.  Mr. 
Long  will  author  the  MR. 

b.  A  Report  of  Test  Results  (RTR),  that  summarizes  the  extent  of  testing  and  results,  will 
start  RW  routing  within  20  working  days  after  the  project  engineering  team  returns  from  the  test. 
Mr.  Long  will  author  the  UH-IN  and  CH-46E  RTR. 

c.  A  final  Project  Briefing,  that  summarizes  the  extent  of  testing  and  results,  will  be  given 
within  20  working  days  after  the  project  engineering  team  returns  from  the  test.  LT.  Hood  and 
Mr.  Long  will  ensure  that  the  briefing  is  conducted. 

d.  Yellow  Sheets,  supplemental  final  reports,  and/or  additional  post-test  briefings  will  be 
completed  within  applicable  time  periods  as  required/desired  by  the  project  sponsor.  Mr.  Long 
will  ensure  that  all  such  post-test  final  reports  or  briefings  are  completed. 

PROJECT  DOCUMENTATION 

64.  As  discussed  in  paragraphs  24  and  56,  test  aircrew  and  DI  department  members  will 
provide  day/night  test  photo  coverage.  The  ship's  flight  deck  video  camera  and  UHF  audio 
recording  systems,  supplemented  by  additional  DI  department  video  camera  systems,  will  be  used 
to  monitor  and  record  all  flight  test  sequences. 

ENVIRONMENTAL  IMPACT 

65.  This  proposed  action  is  viewed  as  a  continuous  test  and  evaluation  mission  activity  that 
poses  no  adverse  threat  to  the  environment;  no  substantial  change  is  occurring  to  the  continuing 
test  and  evaluation  actions  performed  by  FTEG.  No  significant  environmental  degradation  or 
effect  is  known  to  be  occurring  as  a  result  of  these  test  procedures;  therefore,  this  action  is 
considered  not  significant  and  requires  no  further  environmental  documentation. 
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